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Project  Summary 

(Summary  of  research  carried  out  under  ONR  Grant  No.  N00014-91-J-1 137.) 

This  document  serves  as  the  final  technical  report  of  ONR  Grant  No.  NC^14-91-J-1 137. 
Duimilws  project  I  examined  what  people  leant  ftom  examples 

Squaffie  problems.  TheJ&ains  would  be  aieas  such  as  dgb^  probabihty. 

and  ohvsics  A  central  focus  of  this  work  was  to  develop  a  scheme  for  . 

“l^Sg  t£  pmwem  solving  knowledge  that  one  f 

This  was  achieved  using  the  notion  of  subgoals,  which,  m  the  prese  , 

represent  the  task  structure  for  problems  in  a  particular  domam. 

Subeoals  show  the  higher-level  structure  that  can  be  applied  to  a  vmety  of  problems  that 
differ  at  sunerficial  levels  as  weU  as  in  the  mathematical  details  needed  to  solve  tlw 
nmblems  tamers  frequently  focus  on  superficial  aspects  of  examples  and  problems. 

of  steps  that  constitute  the  solution  procedures  to  the  problems. 

A  second  focus  of  the  research  has  been  to  find  ways  of  efff  ti^ly/ony  W  “bgt^Mo 
learners  through  examples  once  those  subgoals  have  been  idenUfied  by 

SSSf^ps  for  achieving  til  subgoal  are  different  in  those  problems  compared  to  the 
example). 

training  and  tutoring  environments. 

The  project  has  yielded  a  number  of  journal  publications,  proceedmgs  papers,  and 
confLence  presentations  that  are  listed  followmg  this  summary. 
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Improving  examples  to  improve 
transfer  to  novel  problems 


RICHARD  CATRAMBONE 
Georgia  Institute  of  Techmlogy,  Atlanta,  Georgia 


People  often  memorize  a  set  of  steps  for  solving  problems  when  they  study  worked^ut  exmpl 
in  domains  such  as  math  and  physics  without  learning  what  domain-relevant  subgoals  or  subtasks 
these  steps  achieve.  As  a  result,  they  have  trouble  solving  novel  problems  that  contain  the  s^e 
structural  elements  but  require  different,  lower-level  steps.  In  three  ei^eriments,  subjects  ^ho  sta¬ 
ted  example  solutions  that  emphasized  a  needed  subgoal  were  more  likely  to  wive  novel  prob 
that  required  a  new  approach  for  achieving  this  subgoal  than  were  subjects  who  did  not  learn  tl^ 
subgoal.  This  result  suggests  that  research  aimed  at  determining  the  factors  that  mfluence  subgoal 
learning  may  be  valuable  in  improving  transfer  from  examples  to  novel  problems. 


A  number  of  studies  have  indicated  that  learners  rely 
heavily  on  worked-out  examples  when  trying  to  solve 
novel  problems  (e.g.,  LeFevre  &  Dixon,  1986;  Pirolli  & 
Anderson,  1985).  Unfortunately,  in  a  domain,  novices 
have  great  difficulty  sepsuating  the  features  of  the  ex¬ 
amples  that  are  necessary  to  the  solution  procedure  from 
those  that  are  incidental  (Ross,  1987, 1989).  In  addition, 
learners  have  difficulty  generalizing  solutions  from  ex¬ 
amples  to  structurally  similar,  but  nonisomorphic,  prob¬ 
lems  (Reed,  Ackinclose,  &  Voss,  1990;  Reed,  Dempster, 
&  Ettinger,  1985).  Although  certain  training  manipula¬ 
tions  have  succeeded  in  improving  transfer  from  exam¬ 
ples  to  novel  problems  to  some  degree  (Lewis  &  Ander¬ 
son,  1985;  Zhu  &  Simon,  1987),  in  general,  transfer  has 
not  been  impressive  (e.g.,  Gick  &  Holyoak,  1983;  Reed 
et  al.,  1985;  Ross,  1987, 1989). 

Learners  differ  in  what  they  extract  from  worked-out 
examples.  Chi,  Bassok,  Lewis,  Reimann,  and  Glaser 
(1989)  found  that  good  learners  are  more  likely  to  try  to 
understand  why  a  particular  step  was  taken  in  a  solution. 
Good  and  poor  learners,  at  least  initially  after  studying  a 
physics  chapter  without  examples,  seemed  to  have  a  sm- 
ilar  level  of  declarative  knowledge  about  mechanics. 
However,  when  studying  an  example,  the  good  learners 
produced  explanations  that  contained  more  “inferences 
about  the  conditions,  the  consequences,  the  goals,  and 
the  meaning  of  various  mathematical  actions  described 
in  the  example”  (p.  168).  Thus,  good  learners  seem  to  get 
more  from  examples,  including  a  knowledge  of  goals. 


This  research  was  supported  by  Office  of  Naval  Research  Grant 
N00014-91-J-1137.  Ilhank  Alana  Anoskey  and  Elinor  Nixon  for  their 
help  in  collecting  and  coding  some  of  the  data.  I  thank  Dedre  Centner, 
Marsha  Lovett,  Brian  Ross,  Tim  Salthouse,  and  Neff  Walker  for  their 
comments  on  earlier  drafts  of  this  paper.  Experiment  I  was  reported 
at  the  Fourth  Annual  Winter  Text  Conference,  January  1993,  Jackwn, 
Wyoming.  Address  correspondence  to  R.  Cattambone,  School  of  Psy¬ 
chology,  Georgia  Institute  of  Technology,  Atlanta,  GA  30332. 


even  when  starting  at  a  knowledge  level  similar  to  that 
of  poor  learners. 

The  present  experiments  explore  whether  more  learn¬ 
ers  can  be  turned  into  good  learners  by  presenting  them 
with  examples  that  convey  the  subgoals  relevant  for 
solving  problems  in  a  domain.  The  term  subgoal  is  used 
here  to  represent  the  task  structure  to  be  learned  for  solv¬ 
ing  problems  in  a  particular  domain  (e.g.,  Catrambone 
&  Holyoak,  1990;  Dixon,  1987;  Eylon  &  R.eif,  1984).  A 
subgoal  groups  a  set  of  steps  under  a  meaningful  task  or 
purpose  (e.g.,  Anzai  &  Simon,  1979;  Chi  &  VanLehn, 
1991 ).  1  hypothesize  that  a  person  who  has  learned  the 
subgoal  will  be  in  a  better  position  to  achieve  it  in  a 
novel  problem  requiring  a  new  or  modified  set  of  steps 
than  w  ill  someone  who  has  not  learned  the  subgoal. 

Learning  subgoals  is  assumed  to  enhance  perfor¬ 
mance  because  subgoals  act  as  ^ides  to  the  part  or  parts 
of  the  procedure  demonstrated  in  examples  that  need  to 
be  changed  for  the  current  problem.  Thus,  subgoals  nar¬ 
row  the  space  in  which  the  solver  has  to  search  in  order 
to  determine  what  must  be  changed.  For  instance,  Simon 
and  Reed  (1976)  found  that  providing  learners  with  a 
subgoal— in  the  form  of  a  hint  to  achieve  a  particular 
state  along  the  solution  path  in  a  problem  involving  mis¬ 
sionaries  and  cannibals — aided  their  navigation  through 
the  problem  space. 

In  the  probability  materials  used  in  the  first  rtvo  ex¬ 
periments,  one  subgoal  toward  the  overall  goal  of  find¬ 
ing  a  particular  probability  is  to  find  the  probability  of 
each  of  the  individual  events.  If  the  steps  for  finding  an 
individual  event  probability  in  a  novel  problem  are  not 
the  same  as  those  used  in  the  example,  then  a  person 
who  has  learned  the  subgoal  to  find  the  individual  event 
probabilities  will  have  a  better  chance  of  focusing  on  the 
steps  of  the  procedure  that  must  be  changed— the  steps 
involved  in  finding  the  individual  event  probabilities— 
than  a  person  who  has  learned  only  a  set  of  steps  for 
finding  the  overall  probability.  For  this  second  learner. 
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the  steps  for  finding  the  individual  probabilities  are  ob¬ 
scured  because  they  are  simply  part  of  a  longer  set  of 
steps  for  reaching  the  end  goal.  This  learner  would  have 
fewer  cues  to  direct  him  or  her  to  the  appropriate  steps 
that  need  to  be  changed. 

There  may  not  be  a  theoretically  “best”  set  of  subgoals 
for  solving  problems  in  a  domain.  The  particular  sub¬ 
goals  that  are  taught  might  represent  an  instructor’s  judg¬ 
ment  about  how  students  should  decompose  problems 
into  subproblems  in  order  to  solve  novel  problems  most 
effectively.  The  judiciousness  of  the  instructor’s  choice 
of  subgoals  can  be  measured  by  the  success  of  the  learn¬ 
ers  on  novel  problems. 

A  learner  will  be  more  likely  to  learn  subgoals  for  the 
subparts  of  a  general  solution  procedure  if  those  sub¬ 
parts  are  emphasized  in  the  examples’  solutions.  This 
claim  hinges  on  the  assumption  that  people  will  form  a 
representation  that  is  based  on  the  most  salient  features 
of  the  example  solution.  The  salience  of  a  feature  will 
vary,  depending  on  the  learner’s  expertise  in  the  domain 
and  how  the  solution  is  presented  (Larkin,  McDermott, 
Simon,  &  Simon,  1980;  Ward  &  Sweller,  1990).  In  the 
present  experiments,  steps  were  labeled  and  visually  sep¬ 
arated  as  a  technique  for  encouraging  the  formation  of  a 
subgoal. 

An  important  first  step  in  creating  useful  examples  is 
to  perform  a  task  analysis  to  determine  what  elements 
need  to  be  learned  in  order  to  solve  problems  in  the  do¬ 
main  of  interest.  How  a  domain  is  analyzed  to  produce 
those  elements  is  by  no  means  standardized.  One  prom¬ 
ising  approach  is  to  create  a  set  of  production  rules  that 
solves  problems  or  carries  out  procedures  that  one  wants 
learners  to  be  able  to  solve  or  learn  (e.g.,  Anderson, 
Boyle,  Farrell,  &  Reiser,  1987;  Kieras  &  Bovair,  1986; 
Zhu  &  Simon,  1987).  Examples  can  then  be  created  in 
order  to  teach  these  productions  to  learners. 

It  does  not  seem  necessary  to  make  a  commitment  to 
a  production  rule  formalism  embodying  a  particular 
learning  theory  such  as  ACT*  (Anderson,  1983)  or  Soar 
(Laird,  Newell,  &  Rosenbloom,  1987)  in  order  to  derive 
the  elements  that  need  to  be  learned.  However,  a  funda¬ 
mental  feature  of  most  production  rule  systems — the 
goal  structure — does  provide  a  useful  way  to  represent 
the  knowledge  needed  to  solve  problems  in  a  domain. 
Subgoals  show  the  breakdown  of  a  problem-solving  pro¬ 
cedure  into  subproblems  (Anzai  &  Simon,  1979).  De¬ 
pending  on  the  features  of  the  examples  and  of  the 
learner,  the  subgoals  learned  from  examples  could  rep¬ 
resent  either  a  flexible  and  general  approach  to  solving 
problems  in  a  particular  domain,  or  a  rigid  and  superfi¬ 
cial  approach. 

Leaniing  Ftom  Exanqtles 

A  number  of  studies  have  shown  that  manipulations 
of  examples  have  a  powerful  and  systematic  effect  on 
performance  on  novel  problems  (e.g.,  Catrambone  & 
Holyoak,  1990;  LeFevre  &  Dixon,  1986;  Pirolli  &  An¬ 
derson,  1985;  Reder,  Charney,  &  Morgan,  1986;  Ross, 
1 984).  Given  the  central  role  that  examples  play  in  prob¬ 
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lem  solving,  and  given  the  assumption  that  people  learn 
subgoals  from  the  examples,  it  is  important  to  investi¬ 
gate  the  conditions  that  influence  subgoal  learning. 

One  reason  that  many  learners  do  not  form  the  “right” 
subgoals  (as  determined  by  an  experimenter  or  instruc¬ 
tor)  is  because  examples  tj^ically  are  not  designed  to 
convey  them.  This  observation  echoes  an  inadequacy  in 
a  mechanics  example  from  a  physics  textbook  noted  by 
Chi  et  al.  (1989,  p.  149).  In  the  example,  a  block  is  sus¬ 
pended  from  a  ceiling  by  two  pieces  of  rope  joined  at  a 
knot  and  a  third  piece  of  rope  extending  from  the  knot 
to  the  block.  The  task  is  to  find  the  magnimde  of  two  of 
the  forces,  given  the  third  force.  The  solution  states  that 
the  knot  where  the  three  strings  are  joined  should  be 
considered  the  body.  However,  no  explanation  is  given 
as  to  why  this  decision  is  made.  The  decision  is  made  be¬ 
cause,  in  order  to  find  a  force  in  terms  of  other  forces, 
the  forces  must  all  act  on  a  common  point.  In  this  prob¬ 
lem,  the  only  place  where  all  three  forces  act  is  the  knot. 
This  critical  subgoal  of  finding  a  common  point  where 
the  forces  are  acting  would  be  useful  for  many  future 
problems.  However,  instead  of  conveying  this  subgoal, 
the  example  is  more  likely  to  convey  a  series  of  steps 
that  may  or  may  not  be  useful  for  other  problems. 

One  question  at  this  point:  Why  not  directly  state  the 
subgoals  to  learners  rather  than  embedding  them  in  ex¬ 
amples?  There  are  two  problems  with  this  approach. 
First,  learners  exhibit  a  clear  preference  for  learning 
from  and  referring  to  examples  when  faced  with  new 
problems  (e.g.,  LeFevre  &  Dixon,  1986;  Pirolli  &  An¬ 
derson,  1 985).  Second,  although  there  have  been  a  small 
number  of  successes  teaching  solution  procedures  di¬ 
rectly  (Fong,  Krantz,  &  Nisbett,  1986),  most  attempts 
have  been  unsuccessful  (e.g.,  Reed  &  Bolstad,  1991). 

Overview  of  Experiments  1  and  2 

In  the  first  two  experiments,  subjects  studied  exam¬ 
ples  that  differed  in  whether  they  emphasized  a  subgoal 
that  was  predicted  to  be  useful  for  solving  novel  test  prob¬ 
lems.  It  was  predicted  that  a  subject  would  be  more  suc¬ 
cessful  at  an  unfamiliar  part  of  a  test  problem  if  he  or  she 
had  learned  the  relevant  subgoal,  compared  with  a  sub¬ 
ject  who  did  not  learn  that  subgoal. 

The  domain  explored  in  Experiments  1  and  2  was 
probability.  This  domain  was  chosen  because  the  train¬ 
ing  and  testing  materials  can  be  relati\ely  simple,  and 
because  prior  work  (Ross,  1987,  1989)  has  provided  a 
useful  manipulation. 

Ross  (1989;  Experiment  IB)  had  subjects  study  ex¬ 
ample  probability  problems,  such  as  ones  involving  per- 
muUtions,  and  then  solve  several  test  problems.  The 
mathematical  roles  of  the  entities  (e.g.,  scientists,  com¬ 
puters)  in  the  examples  and  test  problems  were  manipu¬ 
lated.  For  instance.  Table  1  presents  a  permutation  prob¬ 
lem  involving  the  determination  of  the  probability  that 
scientists  will  pick  particular  computers.  The  equation 
used  for  this  example  was  p  =  l/[n(n-l)...(n-r-l-l)), 
where  n  is  the  number  of  choices  available,  and  r  is  the 
number  of  choices  being  made.  The  test  problems  re- 
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quired  some  subjects  to  find,  for  instance,  the  probabil¬ 
ity  that  students  will  pick  particular  cars  (humans  pok¬ 
ing  inanimate  objects),  while  other  subjects  had  to  find 
the  probability  that  particular  students  would  be  assigned 
to  particular  cars  (i.e.,  objects  “picking  humans). 

Corresponding  mathematical  roles  are  held  by  the  hu¬ 
mans  and  objects  in  the  example  in  Table  1  and  the  test 
problem  involving  students  picking  cars.  In  both  the  ex¬ 
ample  and  the  test  problem,  it  is  the  number  of  humans, 
scientists  and  students,  doing  the  choosing  that  provides 
the  value  for  r.  The  number  of  objects,  computers  and 
cars,  from  which  to  choose  provides  the  value  for  n  The 
second  type  of  test  problem,  cars  “picking”  students, 
however,  has  reversed  object  correspondences;  The  hu¬ 


mans  provide  the  value  for  n  (see  Table  2B  for  another 
example  of  humans  providing  a  value  for  n).  Ross  ( 1 V  sv ) 
found  that  subjects  were  more  successful  at  solving  the 
first  type  of  test  problem  than  the  second,  presumably 
because  their  problem  solving  was  guided  to  some  de¬ 
gree  by  a  feature  correspondence  approach.  Specifically, 
if  the  number  of  objects  provided  the  value  of  «  in  the 
examples,  then  subjects  were  likely  to  assign  them  this 
role  in  a  test  problem,  even  if  it  was  the  number  of  himans 
that  should  have  provided  this  value  in  the  test  problem. 

Besides  working  on  permutation  problems,  Rosss 
subjects  studied  and  solved  combination  problems.  An 
example  combination  problem  might  ask  for  the  proba¬ 
bility  that  the  seven  hooks  nearest  the  classroom  door 


Tablet 

Ptermutation  Training  Example - - - - - 

■nte  aipply  dei»nmM,i  ai  TBM  has  lo  Mke  Inalp^  'kTw 

bers,  respectively^  on  their  computers? 


Table  2 
Test  ProMems 


lavish,  and  fourth  most  lavish  cottages,  respectively? 

B.  Permutation;  Objects  Picking  People  .  .  Ta,H,v  ritv  HaII  received  14  new  chairs,  and  there  are 

The  secretaries  at  city  hall  are  supposed  to  get  ^  tnanagerwants  to  assign  the  chairs  in  the  order  that 

,  1  secretaries  requesting  them.  unpack' dC  rmn^mW  chooses  a  secretary  to  receive  it.  and 

SSn.S'S  »  ha,  U  p..hablh.>  .ha.  .h.  2  «»».«.  .lph.ba.ic.ll,, 

the  first  and  second  chairs  that  are  unpacked,  respectively . 

est  to  the  door  these  smdents  get,  just  as  long  as  it  is  any  I  of  the  7  closest.) 

seats  the  pitchers  get,  just  as  long  as  it  is  any  1  of  the  6  m  the  ftontj — - - - - - - - 


Table  3 

Solutton  Types  Used  for  the  Permutation  Example  In  Table  1 


Subgoal  Solution 

So,  1/1 1  •  1/10«  1/9  =  1/990  •=  overall  probability. 

Equation  Solution 

. ,  u-  U1  i.  1  /-.t  A  -  n.  .(n  -  r  + 1 ).  This  equation  allows  one  to  determine  the  probability 
The  equation  needed  for  this  problem  is  (  ^  .  3  jhe  1 1  represents  the  number  of  computers  that  are 

of  the  above  outcome  occumng.  In  this  problem,  "  J '  being  focused  on  in  this  problem.  The  equa- 
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would  be  picked  by  the  seven  tallest  students  in  a  class 
(see  Table  2C).  The  equation  used  to  solve  combination 
problems  of  this  sort  isp  =  [hlij-hy.],j\.  where  h  is  the 
number  of  entities  (e.g.,  students)  doing  the  choosing, 
and  j  is  the  number  of  entities  in  the  pool  from  which 
things  are  chosen  (e.g.,  hooks).  Again,  Ross  demon¬ 
strated  the  object  correspondence  phenomenon. 

Although  Ross  taught  his  subjects  the  procedures  for 
solving  both  permutation  and  combination  problems  and 
examined  transfer  to  problems  in  which  the  roles  of  hu¬ 
mans  and  objects  were  switched,  an  examination  of  the 
two  procedures  shows  that  a  more  general  procedure  can 
be  used  to  solve  both  problem  types.  Both  permutation 
and  combination  problems  can  be  analyzed  by  consid¬ 
ering  the  individual  event  probabilities  that  contribute  to 
an  overall  probability.  This  approach  is  demonstrated  in 
the  “subgoal”  solution  provided  for  the  problem  of  the 
scientists  and  the  computers  (see  Table  3).  The  combina¬ 
tion  problem,  involving  students  and  coathooks  (Table  2C), 
can  be  analyzed  in  a  similar  way; 

Probability  that  one  of  the  seven  tallest  students  will  get 
a  hook  near  the  door  =7/17. 

Probability  that  one  of  the  remaining  six  tallest  students 
will  get  a  hook  near  the  door  =  6/16. 

Probability  that  one  of  the  remaining  five  tallest  students 
will  get  a  hook  near  the  door  =  5/15,  etc. 

So, 


- - - - -  overall  probability. 

17»16*...*11 

Combination  problems  have  numerators  that  are  no 
longer  simply  “1.”  Instead,  they  start  at  the  number  of 
acceptable  choices  and  then  are  decremented  just  like 
the  denominator. 

The  “subgoal”  solution,  presented  in  Table  3  for  the 
scientists  and  computers  permutation  problem,  is  as¬ 
sumed  to  help  learners  form  two  goals.  The  first  goal  is 
to  find  the  overall  probability;  this  goal  is  assumed  to 
be  formed  because  it  is  explicitly  stated  in  the  example. 
The  second  is  the  subgoal  to  find  each  event  probabil¬ 
ity — for  example,  the  probability  that  the  first  scientist 
will  get  the  computer  with  the  lowest  serial  number,  the 
probability  that  the  second  scientist  will  get  the  com¬ 
puter  with  the  second  lowest  serial  number,  and  so  on. 
This  subgoal  is  assumed  to  be  formed  because  each  in¬ 
dividual  event  probability  is  explicitly  labeled  and  spa¬ 
tially  separate  in  the  subgoal  solution  in  Table  3.  The 
method  for  finding  an  individual  event  probability  will 
involve  the  steps  of  inserting  a  1  in  the  numerator  and 
placing  the  number  of  (remaining)  objects  in  the  denom¬ 
inator  of  each  probability. 

The  “equation”  solution,  presented  in  Table  3  for  the 
scientists  and  computers  permutation  problem,  is  assumed 
to  help  learners  form  only  the  goal  to  find  the  overall 
probability.  The  method  for  achieving  this  goal  will  con¬ 
sist  of  a  set  of  steps  for  finding  numbers  from  the  prob¬ 
lem  statement  and  inserting  them  into  the  equation. 
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Performance  Predictions 

Subjects  who  study  either  the  equation  solution  or  the 
subgoal  solution  are  predicted  to  perform  well  on  the 
first  test  problem  they  are  given  (see  Table  2A).  This 
first  problem  is  a  permutation  problem  that  is  isoinor- 
phic  to  the  training  examples  that  the  subjects  studied, 
and  humans  and  objects  play  the  same  roles  that  were  in 
the  examples  (i.e.,  humans  picking  objects).  The  sub¬ 
jects  can  simply  repeat  the  steps  learned  from  the  ex¬ 
amples.  Performance  on  this  problem  serves  as  a  check 
that  a  subject  has  learned  at  least  a  set  of  steps. 

Two  of  the  three  remaining  test  problems,  one  per¬ 
mutation  problem  and  one  combination  problem  (see  Ta¬ 
bles  2B  and  2D),  reverse  the  roles  for  humans  and  ob¬ 
jects  compared  with  the  training  examples;  that  is,  objects 
are  “picking”  humans.  What  this  means  in  terms  of  the 
solution  is  that  the  numbers  that  go  into  the  denomina¬ 
tor  are  based  on  the  number  of  humans  in  the  problem, 
not  the  number  of  objects.  It  is  predicted  that  subjects 
will  have  difficulty  with  this  aspect  of  the  problems. 
However,  those  who  study  the  subgoal  solution  (the  sub¬ 
goal  group)  are  predicted  to  have  learned  the  subgoal  to 
find  each  individual  event  probability,  and  thus  might  be 
more  likely  to  consider  what  the  numerator  and  denom¬ 
inator  mean  in  each  event  probability.  As  a  result,  these 
subjects  have  a  better  chance  to  consider  modifying  the 
denominator  for  this  problem  than  subjects  in  the  equa¬ 
tion  group. 

The  two  combination  test  problems  (see  Tables  2C 
and  2D)  are  expected  to  cause  difficulty  on  numerator 
performance  since,  unlike  the  examples’  numerators,  the 
numerators  are  no  longer  1 .  Once  again,  the  Subgoal  group, 
by  virtue  of  learning  the  subgoal  to  find  each  indix  idual 
event  probability,  is  predicted  to  outperform  the  equa¬ 
tion  group  because  the  subgoal  of  finding  each  individ¬ 
ual  event  probability  might  lead  the  Subgoal  subjects  to 
consider  the  numerator  as  a  potential  locus  for  change. 

EXPERIMENT  1 

In  Experiment  1, 1  tested  the  hypothesis  that  transfer 
will  be  improved  if  subjects  study  training  examples  that 
emphasize  a  subgoal  needed  for  novel  problems.  Be¬ 
sides  examining  problem-solving  performance,  another 
measure  of  subgoal  learning  was  attempted  by  having 
subjects  describe  how  to  solve  probability  problems 
after  studying  the  examples.  If  a  subject’s  description  in¬ 
cludes  a  statement  such  as  “find  the  probability  of  each 
event’s  occurrence,”  then  this  would  be  taken  as  addi¬ 
tional  support  that  he  or  she  had  learned  that  subgoal. 
The  subgoal  group  should  mention  the  subgoal  of  break¬ 
ing  the  overall  probability  into  a  set  of  individual  event 
probabilities  more  often  than  the  equation  group,  since 
this  is  the  major  difference  in  the  solution  approaches 
presented  to  the  two  groups.  The  subjects  who  mention 
this  subgoal  should  perform  better  than  other  subjects  on 
the  denominators  for  reversed-roles  permutation  and 
combination  problems  and  on  the  numerators  for  the 
combination  problems. 
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Although  the  emphasis  so  far  has  been  placed  on  the 
differences  in  the  solutions  studied  by  the  subgoal  and 
equation  groups,  it  is  important  to  note  that  the  solutions 
are  similar  in  length  and,  presumably,  clarity  (see  Table  3). 
One  could  construct  a  solution  type  that  is  arbitrarily  un¬ 
clear  and  demonstrate  that  subjects’  performance  on  trans¬ 
fer  problems  is  poor  relative  to  subjects  who  study  ex¬ 
amples  using  a  solution  type  that  is  arbitrarily  clear.  It  is 
suggested  that  the  equation  solution  contains  potentially 
useful  information  for  solving  test  problems,  but  that 
subjects  who  study  that  solution  are  likely  to  focus  on 
how  values  are  inserted  into  the  equation  rather  than 
form  a  more  general  procedure. 


Mediod  ^ 

The  subjects  were  66  students  from  mtroductory  psy¬ 
chology  classes  at  the  Georgia  Institute  of  Technology  who  par¬ 
ticipated  for  course  credit.  None  of  the  subjects  had  taken  a  prob- 
ability  course  prior  to  participating  in  the  experiment. 

Materiak  and  Procedure.  The  subjects  received  a  booklet  con- 
teining  training  examples  and  test  problems.  All  the  subjects  stud¬ 
ied  two  isomorphic  worked-out  permutation  example  problems  in 
which  humans  picked  objects.  Table  1  presents  one  of  these 

The  subjects  were  randomly  assigned  to  the  equation  group  (n  = 
31)  or  the  subgoal  group  (n  =  35).  In  the  equation  group  s  exam¬ 
ples  an  equation  was  used  in  order  to  solve  the  problem.  The  so¬ 
lution  included  an  explanation  of  the  meaning  of  the  numbers 
being  inserted  into  the  equation.  The  subgoal  group’s  samples  di¬ 
vided  the  problem  into  finding  each  individual  probability.  Table  3 
contains  the  solutions  that  were  seen  by  the  groups  for  one  of  the 

training  examples,  ^  n  j 

The  subjects  were  asked  to  study  the  examples  carefully  and 
were  told  that,  after  studying  them,  they  would  be  asked  to  solve 
some  problems.  They  were  also  told  that  they  could  not  look  at  the 
examples  when  working  on  the  problems.  This  restriction  was  in¬ 
tended  to  increase  the  likelihood  that  they  would  pay  attention  to 
the  examples  and  how  they  were  solved,  ,  ,  .  ^ 

After  studying  the  examples,  the  subjects  were  asked  to  desenbe 
how  to  solve  problems  in  the  domain.  The  instructions  were:  ‘‘Sup¬ 
pose  you  were  going  to  teach  someone  how  to  solve  probability 
problems  of  the  type  you  have  just  studied.  Please  desenbe  the 
procedure  you  would  give  someone  to  solve  these  problems.  Please 
be  as  complete  as  possible.” 

After  writing  their  descriptions,  the  subjects  attempted  to  solve 
the  four  test  problems  in  Table  2.  The  first  problem  ^ 
phic  to  the  examples  (see  Table  2A).  The  second  problem  (B)  w^ 
a  permutation  problem  like  the  examples,  but  humans  provided  the 
value  of  n  (sec  Table  2B).  The  third  and  fourth  problems  (C  and 
D)  were  combination  problems.  In  the  first  combination  problem, 
humans  picked  objects,  which  is  the  same  notion  shown  m  the  ex¬ 
amples  (see  Table  2C).  In  the  second  combination  problem,  ob¬ 
jects  picked  humans  (see  Table  2D),  Thus,  the  test  problems  rep¬ 
resented  a  range  of  difficulty  as  a  function  of  whether  they 
involved  pcrmuUtions  or  combinations  and  whether  humans  were 

picking  objects,  or  objects  were  picking  humans. 

The  subjects  worked  at  their  own  pace  and  were  asked  to  show 
all  their  work.  In  general,  they  took  about  30  min  to  complete  the 
experiment.  Each  permutation  problem  scored  for  whether  a 
subiect  used  the  correct  denominator.  For  instance,  the  solution  to 
the  second  permutation  problem  is  1/1 1  ♦  1/10.  If  a  subject  wrote 
1/14*1/13,  confusing  the  roles  of  the  chairs  and  secrctanes,  the 
denominator  would  be  scored  as  incorrect.  For  combination  prob¬ 
lems,  the  numerator  and  denominator  were  both  scored  as  correct 
or  incorrect.  Two  raters  independently  scored  the  problems;  mcir 
scores  agreed  92%  of  the  time.  Any  disagreements  were  resolved 


by  discussion.  The  frequencies  with  which  the  groups  correctly 
found  the  denominators  for  the  permutation  problems  and  the  de¬ 
nominators  and  numerators  for  the  combination  problems  were  an¬ 
alyzed  by  using  the  likelihood  ratio  chi-square  test  (C^;  Bishop, 
Fienberg,  &  Holland,  1975). 

Results  and  Discussion 

The  overall  performance  differences  between  the 
groups  can  be  summarized  as  follows;  They  did  not  re- 
Fiably  differ  on  denominator  performance  for  the  reversed- 
role  problems  (i.e.,  objects  picking  humans),  but  the 
subgoal  group  outperformed  the  equation  group  on  the 
numerators  for  the  combination  problems. 

As  expected,  both  groups  were  quite  successful  in  de¬ 
termining  the  denominator  on  the  first  permutation 
problem,  which  was  isomorphic  to  the  training  examples 
and  had  humans  and  objects  playing  the  same  roles  as  m 
the  examples  [G^fl)  =  0.5,  p  =  .78;  see  Table  4].  ^ 

In  the  second  permutation  problem,  objects  picked  hu¬ 
mans — a  reversal  from  the  training  examples.  An  error 
that  the  subjects  frequently  made  on  this  problem  was  to 
use  the  number  of  chairs  (14)  as  the  starting  point  in  the 
denominator  rather  than  the  number  of  secretaries  (1 1 ). 
Although  the  groups  did  not  significantly  differ  in  find¬ 
ing  the  denominator  [0^(1)  =  1.1 1,P  =  .29],  there  was 
an  1 1%  advantage  for  the  subgoal  group  (see  Table  4). 

The  next  problem  was  a  combination  problem  m 
which  humans  picked  objects.  The  subjects  were  fairly 
successful  at  finding  the  denominator  and,  as  expwted, 
did  not  differ  significantly  on  this  measure  7  09, 

p  =  .76;  see  Table  4].  However,  they  did  have  difficulty 
finding  the  correct  numerator  for  this  problem;  many  of 
the  subjects  simply  used  1.  As  expected,  the  subgoal 
group  outperformed  the  equation  group  on  this  measure 
[G2(1)  =  6.39,p  =  .01]. 

In  the  second  combination  problem,  objects  picked 
humans.  As  in  the  reversed-roles  permutation  problem, 
the  subjects  had  difficulty  finding  the  correct  value  for 
the  denominator.  The  groups  did  not  differ  significantly 
■  on  this  measure  [G2(l)  =  .06,p  =  .80;  see  Table  4],  As 
in  the  first  combination  problem,  the  subjects  had  diffi- 
'  culty  finding  the  correct  numerator.  As  expected,  Ae 
subgoal  group  was  more  successful  than  the  equation 
•  group  on  this  measure  [G^(l)  =  9.83,  p  =  .002]. 


lM)le4 

fta'foniunce(Fereeiit  Correct)  on  Experiment  1  Test  ProMems 

_ Group _ 

Equation  Subgoal 

_ (n  =  31) _ ("  ==  35) 

Permutation  Problem  1  (people  pick  objects) 

Denominator 

Permutation  Problem  2  (objects  pick  people) 

Denominator  23 

Combination  Problem  1  (people  pick  objects) 

Denominator  21 

Numerator  I  ^ 

Combination  Problem  2  (objects  pick  people) 

Denominator  23 

Numerator  _ _ _ - - - — 
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The  transfer  results  suggest  that  the  subjects  in  both 
groups  were  equally  misled  by  superficial  role  reversals 
of  objects  and  humans  in  the  denominator.  However,  the 
subgoal  subjects  were  more  likely  to  adapt  their  solution 
procedure  to  find  the  numerator  correctly  in  combina¬ 
tion  problems. 

Rdationship  between  training  examples  and  expla¬ 
nations  produced  by  subjects  on  how  to  solve  probleim. 
The  subjects’  explanations  of  how  to  solve  problems  in 
the  domain  were  scored  for  whether  they  mentioned  the 
subgoal  of  dividing  the  overall  probability  into  a  series 
of  individual  probabilities.  Six  equation  subjects  and  5 
subgoal  subjects  produced  explanations  that  were  too 
general  or  idiosyncratic  to  be  scored.  These  explanations 
typically  consisted  of  statements  such  as  “1  would  read 
through  the  example  and  write  it  up  on  the  board  for  the 
person.”  These  subjects  were  excluded  from  the  follow¬ 
ing  analyses. 

The  subgoal  group  mentioned  the  notion  of  dividing 
the  overall  probability  into  a  set  of  individual  probabil¬ 
ities  far  more  often  than  the  equation  group  (83%  vs. 
8%;  G2(l)  =  35.3, p  <  .0001].  This  was  expected,  since 
the 'subgoal  group  studied  example  solutions  that  labeled 
and  isolated  individual  probabilities,  whereas  the  equa¬ 
tion  group  did  not. 

Relationship  between  explanations  and  transfer  per¬ 
formance.  It  was  expected  that  the  subjects  who  men¬ 
tioned  the  subgoal  of  finding  individual  event  probabil¬ 
ities  in  their  explanations  would  be  more  likely  to 
correctly  find  the  denominator  in  the  reversed-role  prob¬ 
lems  and  the  numerator  in  the  combination  problems. 

There  was  no  difference  in  denominator  performance 
for  the  first  permutation  problem  for  the  subjects  who 
mentioned  the  notion  of  dividing  a  probability  into  in¬ 
dividual  probabilities  (the  “IndProb”  subjects)  com¬ 
pared  with  the  subjects  who  did  not  mention  this  notion 
in  their  explanations  (the  “OneProb”  subjects)  (G^l)  = 
.18,  p  =  .67;  see  Table  5].  This  is  not  suiprising,  since  in 
this  problem  humans  choose  objects,  as  in  the  examples. 


Tables 

rtrlbniiaiice  (Percent  Correct)  on  Test  ProWeim 

ns  a  Function  of  Subjects’  Explanations  in  Experiment  1 

_ Group _ 

OneProb*  IndProbf 

_ (n  27) _ in  ^  28) 

Permutation  Problem  1 
Denominator 
Permutation  Problem  2 
Denominator 
Combination  Problem  1 
Denominator 
Numerator 

Combination  Problem  2 

Denominator  11  ^ 

Numerator _ _ _ IJ _ _ _ ^  _ 

*Docs  not  mention  breaking  problem  into  individual  probabilities. 
tMcntions  breaking  problem  into  individual  probabilities. 


As  expected,  the  IndProb  subjects  were  more  suc¬ 
cessful  than  the  OneProb  subjects  at  finding  the  correct 
value  for  the  denominator  in  the  second  permutation 
problem,  which  involved  reversed  roles  for  humans  and 
objects  [GK\)  =  3.84,p  =  .05;  see  Table  5]. 

In  the  first  combination  problem,  humans  and  objects 
played  the  same  roles  that  were  in  the  examples,  and 
therefore  it  is  not  surprising  to  find  that  there  was  no  dif¬ 
ference  between  the  groups  in  finding  the  correct  value 
for  the  denominator  in  this  problem  [G-(l)  =  .\5,  p  = 
.69;  see  Table  5].  As  expected,  the  IndProb  subjects  were 
more  successful  than  the  OneProb  subjects  at  finding  the 
conect  value  for  the  numerator  [G^l)  =  6.26,  p  =  .01 ; 
see  Table  5]. 

Although  the  anticipated  difference  between  the 
groups  in  finding  the  correct  value  for  the  denominator 
in  the  second  combination  problem — a  reversed-roles 
problem — ^was  not  found  [G^(l)  =  2.18,  p  =  .14],  the 
IndProb  group  had  a  15%  advantage  (see  Table  5).  As 
expected,  the  IndProb  subjects  were  more  successful 
than  the  OneProb  subjects  at  finding  the  correct  numer¬ 
ator  value  [G2(l)  =  8.29,p  =  .004;  see  Table  5]. 

These  results  follow  the  trend  of  those  that  were  found 
when  the  instructional  groups  were  compared  on  the 
transfer  problems.  This  makes  sense,  since  the  subgoal 
subjects  were  by  far  the  ones  most  likely  to  mention  the 
notion  of  dividing  an  overall  probability  into  individual 
probabilities. 

It  could  be  argued  that  learners  who  write  “better”  de¬ 
scriptions  (e.g.,  mention  the  subgoal  of  finding  individ¬ 
ual  event  probabilities)  are  also  the  ones  who  are  better 
at  transfer.  One  defense  against  this  argument  is  to  note 
that  the  experimental  manipulation  of  type  of  examples 
studied  influenced  transfer  success. 

Although  the  subgoal  subjects  did  not  reliably  out¬ 
perform  the  equation  subjects  on  the  denominators  for 
leversed-role  problems,  they  were  clearly  superior  at  find¬ 
ing  the  correct  numerators  for  the  combination  problems. 
In  addition,  the  IndProb  group  outperformed  the  OneProb 
group  on  finding  the  denominator  for  one  of  the  reversed- 
role  problems.  Although  most  of  the  results  support  the 
predictions,  the  unreliability  of  the  denominator  effect 
for  the  reversed-role  problems  suggested  that  a  second, 
more  focused  experiment  would  be  appropriate. 

EXPERIMENT  2 

The  procedure  and  materials  for  Experiment  2  were 
identical  to  those  of  Experiment  1,  except  for  three  fea¬ 
tures:  (1)  the  subjects  studied  three  rather  than  two  train¬ 
ing  examples,  (2)  the  subjects  were  not  asked  to  write 
explanations  of  how  to  solve  problems,  and  (3)  for  half 
of  the  subjects,  the  combination  problems  did  nof  con¬ 
tain  the  last  sentence  shown  for  each  combination  prob¬ 
lem  in  Table  2. 

The  number  of  examples  presented  to  the  subjects  was 
increased  to  improve  the  likelihood  that  they  would 
learn  the  procedures  demonstrated  in  the  examples  and 
perhaps  improve  transfer  to  the  test  problems.  They  were 
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not  asked  to  write  explanations,  since  this  task  was  un¬ 
usual  and  may  have  influenced  learning  in  unanticipated 
ways.  Finally,  the  last  sentence  for  each  combination 
problem  in  Table  2  was  removed  for  half  of  the  subjects 
in  order  to  examine  whether  they  signaled  to  the  subjects 
that  these  problems  were  different  from  the  training  ex¬ 
amples  and  needed  to  be  approached  differently.  Perhaps 
with  this  cue  removed,  the  subgoal  subjects  would  not 
outperform  the  equation  subjects. 

Method 

Subjects.  The  subjects  were  78  students  from  introductory  psy¬ 
chology  classes  at  the  Georgia  Institute  of  Technology  who  par¬ 
ticipated  in  the  experiment  for  course  credit.  None  of  the  subjects 
had  taken  a  probability  course  prior  to  participating  in  the  exper¬ 
iment. 

Materials  and  Procedure.  The  subjects  received  a  booklet  con¬ 
taining  training  examples  and  test  problems.  They  all  studied  three 
isomorphic  worked-out  permutation  example  problems  in  which 
humans  picked  objects  (including  the  two  used  in  Experiment  1). 

The  subjects  were  randomly  assigned  to  the  equation  group 
(„  =  40)  or  the  subgoal  group  (n  =  38).  After  studying  the  exam¬ 
ples,  they  solved  the  four  test  problems  in  Table  2.  The  subjects 
worked  at  their  own  pace  and  were  asked  to  show  all  their  work. 
In  general,  they  took  about  25  min  to  complete  the  experiment. 
Two  raters  independently  scored  the  problems,  and  their  scores 
agreed  90%  of  the  time.  Any  disagreements  were  resolved  by 
discussion. 

Results  and  Discussion 

The  results  were  similar  to  those  from  Experiment  1 
and  supported  most  of  the  predictions.  The  subgoal  group 
strongly  outperformed  the  equation  group  on  the  nu¬ 
merators  for  the  combination  problems  and  showed  a 
trend  toward  superior  performance  on  the  denominator 
for  one  of  the  two  reversed-role  problems. 

As  expected,  both  of  the  groups  did  well  in  determin¬ 
ing  the  denominator  for  the  first  permutation  problem, 
and  there  was  no  significant  difference  in  their  perfor¬ 
mance  [G2(l)  =  0.63, />  =  .43;  see  Table  6].  As  in  Ex¬ 
periment  1,  both  of  the  groups  showed  inferior  perfor¬ 
mance  in  determining  the  denominator  for  the  second 
permutation  problem— a  problem  with  reversed  roles  for 
humans  and  objects.  As  predicted,  there  was  an  advan¬ 
tage  (20%)  for  the  subgoal  group,  although  this  differ¬ 
ence  just  missed  significance  [G^Cl)  =  3.47,  p  -  .06; 
see  Table  6].  .  . 

Performance  on  the  combination  problems  was  ini¬ 
tially  broken  down  as  a  function  of  training  group  and 
whether  or  not  the  problems  contained  the  last  sentence 
presented  for  each  combination  problem  in  Table  2.  There 
was  no  main  effect  of  sentence  for  either  problem,  nor 
was  there  an  interaction  between  group  and  presence  of 
the  sentence;  thus,  the  analyses  and  the  results  for  the 
combination  problems  are  collapsed  across  this  factor  in 
Table  6. 

In  the  first  combination  problem,  humans  picked  ob¬ 
jects,  and,  as  anticipated,  the  subjects  were  fairly,  and 
equally,  successful  at  finding  the  denominator  [G^l)  = 
.009,  p  -  .92;  see  Table  6].  As  expected,  the  subgoal 


Table6 

Pferfomuince  (Percent  Correct)  on  Experiment  2  Test  Problems 

_ Group _ 

Equation  Subgoal 

(h  =  40) _ (w  =  38) 

Permutation  Problem  1 
Denominator 
Permutation  Problem  2 
Denominator 
Combination  Problem  1 
Denominator 
Numerator 

Combination  Problem  2 
Denominator 
Numerator 


subjects  outperformed  the  equation  subjects  at  finding 
the  correct  numerator  [G2(l)  =  17.7,  /?  =  .0001;  see 
Table  6]. 

The  predicted  superior  performance  by  the  subgoal 
subjects  did  not  occur  for  finding  the  correct  value  for 
the  denominator  in  the  second  combination  problem,  a 
reversed-roles  problem  [G^(l)  —  0.02,  p  —  .88;  see 
Table  6].  As  expected,  the  subgoal  subjects  outper¬ 
formed  the  equation  subjects  at  finding  the  correct  nu¬ 
merator  [G^l)  =  19.6, p  =  .0001;  see  Table  6]. 

It  was  hypothesized  that  the  subgoal  subjects  learned 
the  subgoal  to  find  individual  event  probabilities,  but  the 
equation  subjects  did  not.  This  subgoal  was  predicted  to 
aid  performance  in  finding  the  denominator  for  reversed- 
role  problems  and  the  numerators  for  combination  prob¬ 
lems.  Both  experiments  clearly  demonstrated  the  numer¬ 
ator  effect.  Experiment  1  demonstrated  a  trend  toward 
the  denominator  effect  for  the  reversed-role  problems, 
primarily  when  the  subjects  were  partitioned  into  those 
who  mentioned,  or  failed  to  mention,  the  subgoal  of 
finding  individual  event  probabilities  in  their  explana¬ 
tions.  This  trend  was  stronger  for  the  permutation  prob¬ 
lem  than  for  the  combination  problem.  Experiment  2 
also  showed  this  effect  more  strongly  for  the  permuta¬ 
tion  problem.  It  is  not  clear  if  there  are  certain  features 
of  these  two  problems  that  differentially  affected  de¬ 
nominator  performance.  The  subjects’  explanations  of 
how  to  solve  problems  were  consistent  with  the  claim 
that  the  subgoal  subjects  were  more  likely  to  learn  the 
subgoal  to  find  individual  event  probabilities  than  the 
equation  subjects. 

The  fact  that  the  denominator  effect  was  less  reliable 
than  the  numerator  effect  may  be  due  to  the  role  of  su¬ 
perficial  features.  The  training  examples  presumably  led 
the  subjects  to  expect  objects  to  provide  the  value  for  the 
denominator,  and  perhaps  this  expectation  tended  to 
override  any  benefits  due  to  learning  subgoals,  espe¬ 
cially  since  a  value  for  objects  was  always  provided  in 
each  problem.  The  tendency  to  put  a  1  as  the  numerator 
may  have  been  more  easily  overridden  by  subgoal  learn¬ 
ing  because  its  connection  with  either  humans  or  objects 
is  less  clear. 
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EXPERIMENTS 

Experiment  3  was  an  attempt  to  generalize  the  find¬ 
ings  from  the  first  two  experiments  to  another  domain; 
algebra  word  problems.  This  domain  was  chosen  be¬ 
cause  prior  work  had  demonstrated  poor  transfer  from 
training  to  transfer  problems  despite  attempts  to  improve 
examples  (e.g.,  Reed  et  al.,  1985).  Consider  the  algebra 
example  in  Table  7A,  in  which  one  has  to  determine  how 
long  it  would  take  someone  to  do  a  job  given  that  cer¬ 
tain  information  about  their  work  rate  and  time  and  an¬ 
other  person’s  work  rate  and  time  are  provided.  This 
problem  involves  using  an  equation  for  determining 
work  that  requires  representing  each  workerls  work  rate 
and  time;  (rate,  X  time,)  +  (ratej  x  timej)  =1. 

Learners  are  good  at  memorizing  how  to  solve  prob¬ 
lems  that  are  isomorphic  to  the  one  in  Table  7A.  In  this 
problem,  both  of  the  workers’  rates  are  represented  as 
constants.  The  time  spent  working  by  Worker  1  is  rep¬ 
resented  as  a  variable,  and  Worker  2’s  time  is  repre¬ 
sented  as  a  function  of  that  variable.  However,  l^ers 
may  not  encode  the  example  solution  by  determining  a 
representation  for  each  rate  and  time  and  then  inserting 
these  representations  into  the  equation.  Instead,  they 
have  a  more  superficial  understanding  of  the  solution 
procedure,  which  involves  matching  the  form  used  in 
the  example,  finding  similar  values  in  the  problem 
statement,  and  inserting  them  into  the  equation.  As  a  re¬ 
sult,  if  a  new  problem  requires  a  different  representation 
of  the  rates  and  times,  these  learners  may  be  unable  to 
solve  the  problem.  That  is,  the  learners  may  not  have 
learned  that  certain  subgoals  exist — the  subgoals  of  rep¬ 
resenting  each  worker’s  rate  and  time — and  that  these 
subgoals  might  be  achieved  differently  (i.e.,  different 
ways  of  representing  rate  and  time  depending  on  the 
givens  in  the  problem)  from  the  way  they  were  achieved 
in  the  example. 

For  instance,  the  problem  in  Table  7B  requires  that 
Worker  1  ’s  rate  be  represented  as  a  variable.  In  addition, 
inctpaH  of  having  the  workers’  times  be  represented  as  a 
variable  and  a  function  of  that  variable,  the  times  are  rep¬ 
resented  as  a  constant  and  a  function  of  that  constat. 
Nevertheless,  the  new  representations  can  be  inserted  into 
the  same  equation  that  was  used  for  the  example  in 
Table  7A.  Similarly,  the  problem  in  Table  7C  requires 
that  Worker  I’s  rate  be  represented  as  a  variable,  and 
Worker  2’s  rate  be  a  function  of  that  variable.  Their  times 
are  both  represented  as  constants.  These  representations 
are  different  from  those  used  in  the  examples. 

It  is  hypothesized  that  if  the  representations  for  rates 
and  times  are  highlighted  separately  from  the  equation 
in  the  example  solutions,  then  learners  will  be  more  likely 
to  learn  that  rate  and  time  are  individual  representations 
diat  must  be  determined  for  each  worker.  In  addition,  it 
is  hypothesized  that  if  subjects  learn  the  subgoals  of  rep¬ 
resenting  workers’  rates  and  times,  then  they  will  be 
more  likely  to  correctly  solve  a  novel  problem  requiring 
novel  representations  for  rates  and  times  than  would  sub¬ 
jects  vriio  do  not  learn  those  subgoals. 


TiMe? 

p«mple«ndT«tWBrklVi)btoMinExperiin«nt3 

A.  Mary  can  rebuild  a  carburetor  in  3  hours  and  Mike  can  rebuild  one 
in  4  hours.  How  long  would  it  take  Mary  to  rebuild  a  carburetor  if  she 
and  Mike  work  together,  but  Mike  works  for  '/j  hour  more  than  Mary? 

Solution 

1/3  =  Mary’s  rate 

t  -  time  Mary  spent  rebuilding  carburetor 
1/4  Mike’s  rate 

/  +  1/2  =  time  Mike  spent  rebuilding  carburetor 
(1/3*0+ [l/4*(/+ 1/2)]  =  1 
7/12*/=l-l/8 

f  =  7/8*12/7  =  3/2  hours  =  time  Mary  spent  rebuilding  carburetor 

B.  Mr.  Jones  can  refinish  a  dresser  in  5  hours.  After  working  for  2 
hours,  he  is  joined  by  his  wife.  Together  they  finish  the  job  in  1  hour. 
How  much  of  the  job  could  his  wife  do  in  1  hour  when  working  alone? 

Solution  [not  seen  by  subjects] 

[l/5*(2+l)]  +  (w*l)  =  l 

3/5  +  w=l 

w  =  2/5  =  wife’s  rate 

so,  in  1  hour,  wife  could  do  2/5  of  job 

C.  Barbara  and  Connie  can  finish  a  job  in  6  hours  when  they  work  to¬ 
gether.  Barbara  works  twice  as  fast  as  Connie.  How  much  of  the  job 
could  Connie  do  in  1  hour  when  working  alone? 

Solution  [not  seen  by  subjects] 

(2c*6)  +  (c*6)  =  1 
12c  +  6c  =  1 
18c=  1 

c  =  1/18  =  Connie’s  rate 

so,  in  1  hour,  Connie  could  do  1/18  of  job  _ 

Subjects  who  leam  the  subgoal  to  represent  each 
worker’s  rate  should  be  more  successful  at  representing 
the  rate  as  a  variable  in  the  problem  in  Table  7B  and  rate 
as  a  variable  and  rate  as  a  function  of  a  variable  in  the 
problem  in  Table  7C.  Subjects  who  leam  the  subgoal  to 
represent  each  worker’s  time  should  be  more  successful 
at  representing  the  time  as  a  constant  and  a  function  of 
a  constant  in  the  second  problem  (B)  and  as  a  constant 
in  the  third  problem  (C). 

Method 

Subjects.  Sixty-two  students  from  introductory  psychology 
classes  at  the  Georgia  Institute  of  Technology  participated  for 
course  credit. 

Materials  and  Procedure.  The  subjects  studied  three  isomor¬ 
phic  example  word  problems  dealing  with  work,  including  the 
example  in  Table  7A.  The  “rate  and  time  label”  (RTL)  group  («  = 
21)  studied  examples  in  which  the  representations  for  rates  and 
times  were  presented  separately  from  the  equation  (sec  Lines  1-4 
in  the  solution  to  the  example  in  Table  7A).  The  ‘‘time  label”  (TL) 
group  (n  =  20)  studied  examples  that  presented  the  representa¬ 
tions  for  each  worker’s  time  (i.e..  Lines  2  and  4  from  the  example 
in  Table  7  A).  The  “rate  label”  (RL)  group  (n  =  2 1 )  studied  exam¬ 
ples  that  presented  the  representations  for  each  worker’s  rate  (i.e., 
Lines  1  and  3  from  the  example  in  Table  7A). 

After  studying  the  examples,  the  subjects  received  three  prob¬ 
lems  to  solve.  One  was  isomorphic  to  the  training  examples,  and 
the  other  two  involved  new  and  old  ways  of  representing  rate 
and/or  time  (sec  Tables  7B  and  7C). 

Results  and  Discussion 

All  of  the  groups  performed  well  at  representing  rate 
and  time  in  Problem  1,  which  was  isomorphic  to  the 
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Tables 

Btribniiaiice(lV»tantCbrrect)onT«tProMen>sinE«periiiient3 


RTL 
(n  =  21) 

Group 

TL 

(n  =  20) 

RL 

("  “  21) 

Problem  1 

Rate  Worker  1  (constant) 

100 

100 

100 

Rate  Worker  2  (constant) 

100 

100 

95 

Time  Worker  1  (variable) 

100 

100 

100 

Time  Worker  2 

95 

95 

100 

(function  of  a  variable) 

Problem  2 

Rate  Worker  1  (constant) 

100 

95 

100 

Rate  Worker  2  (variable) 

86 

55 

86 

Time  Worker  1 

81 

90 

57 

(function  of  a  constant) 
Time  Worker  2  (constant) 

90 

90 

57 

Problem  3 

Rate  Worker  1  (variable) 

90 

60 

90 

Rate  Worker  2 

86 

50 

86 

(function  of  a  variable) 

Time  Worker  1  (constant) 

90 

100 

62 

Time  Worker  2  (constant) 

90 

100 

62 

Note — RTL,  tate  and  time  label;  TL,  time  label;  RL,  rate  label. 


training  examples  (see  Table  8).  This  is  not  surprising, 
since  the  subjects  could  match  the  representations  from 
the  training  examples  and  simply  plug  in  the  new  values. 

In  Problem  2  (Table  8),  almost  all  of  the  subjects  cor¬ 
rectly  represented  the  rate  as  a  constant  for  Worker  1. 
Again,  this  is  reasonable  since  this  was  the  representa¬ 
tion  used  in  the  training  examples.  The  RTL  and  RL 
groups  were  significantly  more  successful  than  the  TL 
group  at  representing  the  rate  for  Worker  2  as  a  variable 
[G2(2)  =  6.63,  p  =  .04].  The  RTL  and  TL  groups  were 
more  successful  than  the  RL  group  at  representing  the 
rate  for  Worker  2  as  a  constant  [G^(2)  -  8.78,  />  =  .01] 
and  Worker  1  as  a  function  of  that  constant  [G2(2)  = 
6.50,p=  .04]. 

In  Problem  3  (Table  8),  the  RTL  and  RL  groups  were 
significantly  more  successful  than  the  TL  group  at  rep¬ 
resenting  the  rate  for  Worker  1  as  a  variable  [G2(2)  = 
7.59,  p  =  .02]  and  Worker  2  as  a  function  of  that  vari¬ 
able  [G2(2)  =  8.63,  p  =  .01].  The  RTL  and  TL  groups 
were  more  successfiil  than  the  RL  group  at  representing 
the  rates  for  Workers  1  and  2  as  constants  [G^(2)  =  13.66, 
p  =  .001,  in  both  cases].  Across  the  problems,  the  most 
common  errors  were  that  the  subjects  either  left  out  the 
representation  for  time  or  rate  in  the  equation,  or  wrote 
that  not  enough  information  was  given  in  the  problem. 

The  results  from  this  experiment  are  consistent  with 
the  hypothesis  that  subjects  who  learned  the  subgoals  of 
representing  workers’  rates  and  times  would  represent 
them  more  successfully  on  novel  problems.  However,  al¬ 
ternate  explanations  exist.  One  is  that  the  highlighting 
manipulation  essentially  provided  labels  for  the  vari¬ 
ables,  and  thus  made  them  more  meaningful.  This  “mean¬ 
ingfulness”  helped  the  subjects  to  properly  use  the  vari¬ 
ables  in  the  equation  for  the  novel  transfer  problems.  The 
results  from  Experiment  3,  taken  alone,  are  not  sufficient 
to  discriminate  between  this  explanation  and  the  subgoal 

Page  12 


explanation.  However,  the  pattern  of  results  across  the 
three  experiments  is  consistent  with  a  subgoal  explana¬ 
tion.  It  is  not  clear  how  the  “subpal”  solution  in  the  first 
two  experiments  made  the  variables  more  meaningful. 

One  could  consider  the  subgoal  explanation  of  the  re¬ 
sults  as  being  an  attempt  to  make  the  notion  of  mean- 
ingfiilness  more  precise.  Subgoals  can  be  viewed  as  a 
way  of  making  a  solution  procedure  more  meaningful  by 
providing  guideposts  that  the  solver  must  reach  en  route 
to  achieving  the  overall  solution  to  the  problem.  In  Ae 
algebra  experiment,  the  guideposts  were  to  explicitly 
represent  the  rate  and  time  for  each  worker.  Even  if  these 
guideposts  are  somewhat  arbitrary,  at  least  from  the 
point  of  view  of  the  solver,  they  still  provide  organiza¬ 
tion  and  guidance  that  may  make  the  solver  less  likely 
to  stray  from  the  correct  solution  path  (cf.  Mawer  & 
Sweller,  1982). 

It  is  worth  noting  that  performance  on  the  test  prob¬ 
lems  was  generally  quite  good.  This  suggests  that  even 
for  relatively  sophisticated  subjects  (most  of  the  students 
in  the  experiment  had  at  least  one  term  of  college  cal¬ 
culus),  examples  that  more  effectively  convey  subgoals 
can  improve  transfer  performance. 

GENERAL  DISCUSSION 

The  aim  of  the  present  study  was  to  examine  whether 
examples  that  teach  a  subgoal  structure  for  solving  prob¬ 
lems  in  a  domain  could  be  created  and  whether  learning 
these  subgoals  would  help  subjects  solve  problems  that 
required  novel  methods  for  them.  The  numerator  per¬ 
formance  results  and  some  of  the  denominator  perfor¬ 
mance  results  in  the  probability  experiments,  and  the 
rate  and  time  results  in  the  algebra  experiment,  suggest 
that  subgoals  can  be  conveyed  to  learners  through  ex¬ 
amples,  and  that  learning  these  subgoals  helps  people 
achieve  them  in  novel  problems.  This  is  quite  encourag¬ 
ing  in  comparision  with  the  usual  finding  in  the  prob¬ 
lem-solving  literature,  which  has  shown  poor  transfer 
from  training  materials  to  test  problems  that  require 
more  than  a  simple  repetition  of  a  set  of  memorized 
steps  (e.g.,  Reed  et  al.,  1985;  Ross,  1987, 1989).  These 
findings  suggest  that  examples  that  emphasize  a  useful 
subgoal  structure  can  help  turn  learners  into  the  “good” 
learners  observed  by  Chi  et  al.  (1989),  who  tended  to 
find  meaning,  such  as  goals,  for  the  mathematical  steps 
in  the  examples  and  who  made  use  of  this  information 
when  solving  novel  problems. 

Mayer  and  Greeno  (1972)  experimentally  manipu¬ 
lated  the  meaningfulness  of  instruction  in  solving  bi¬ 
nomial  probability  problems  by  varying  whether  the 
instruction  focused  on  mechanical  operations  or  on  con¬ 
cepts  that  were  presumed  to  be  part  of  subjects’  prior 
knowledge.  They  found  that  the  “mechanical”  group 
was  more  successful  at  solving  familiar  problems, 
whereas  the  “concept”  group  was  better  at  answering 
“understanding”  questions  about  the  domain,  such  as 
whether  the  number  of  successes  could  be  greater  than 
the  number  trials.  The  present  study  extends  Mayer  and 
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Greeno’s  findings  by  showing  that  learning  subgoals 
promotes  transfer  to  novel  problems  w^hile  also  helping 
learners  exhibit  some  level  of  understanding,  as  sho\^m 
by  the  explanations  produced  by  the  subgoal  subjects  in 
Experiment  1. 

CONCLUSIONS 

The  present  results  are  consistent  with  the  claim  that 
learning  subgoals  will  help  learners  determine  which 
parts  of  a  solution  procedure  need  to  be  modified  in 
order  to  solve  novel  problems.  The  results  also  provide 
a  starting  point  for  determining  how  to  “emphasize”  a 
subgoal  in  an  example.  The  approach  used  here  was  to 
create  example  solutions  that  isolated  components  of  the 
procedure  that  could  be  construed  as  subgoals.  The  vi¬ 
sual  separation  and  the  labeling,  and  perhaps  their  in¬ 
teraction,  may  have  all  played  a  role  in  subgoal  learning. 

Smith  and  Goodman  (1984)  examined  subgoal  learn¬ 
ing  by  comparing  a  group  of  subjects  who  followed  a  set 
of  steps  for  assembling  an  electric  circuit  with  a  group 
who  received  a  structurally  oriented  “explanatory  schema” 
with  the  steps.  This  schema  consisted  of  statements  that 
provided  a  rationale  for  carrying  out  sets  of  steps.  Each 
rationale  was  essentially  a  statement  of  a  goal  that  the 
steps  were  achieving  (e.g.,  “The  next  thing  that  you  will 
have  to  do  is  to  assenTble  the  on-off  switch”),  ^^en  as¬ 
sembling  a  new  circuit,  the  subjects  who  had  previously 
received  the  explanatory  schema  were  more  accurate  at 
building  the  substructures  corresponding  to  the  goals, 
even  though  the  required  steps  were  not  identical  to  the 
ones  followed  during  training. 

The  present  results,  and  those  from  Smith  and  Good¬ 
man  (1984),  suggest  that  research  aimed  at  determining 
factors  that  affect  subgoal  learning,  such  as  the  use  of  la¬ 
bels  and  visual  separation  of  steps  in  examples,  would 
have  clear  pedagogical  benefits.  In  addition,  it  is  im¬ 
portant,  both  in  terms  of  theory  development  and  the 
production  of  effective  training  materials,  to  explore 
what  constitutes  a  good  subgoal  structure  for  a  given  do¬ 
main.  Perhaps  a  theory-motivated  technique  can  eventu¬ 
ally  be  developed  for  determining  effective  subgoal 
structures  for  any  given  domain. 
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Students  often  memorize  a  set  of  steps  from  examples  in  domains  such  ds  probability  znd 
physics,  without  learning  what  subgoals  those  steps  achieve.  A  result  of  this  sort  of  learning 
can  be  that  these  students  fail  to  solve  novel  problems  that  do  not  permit  exactly  the  same  set 
of  steps  even  though  the  old  goal  structure  is  maintained.  Three  experiments  demonstrated 
that  both  labeling  and  visually  isolating  a  set  of  steps  in  examples  independently  help  students 
learn  a  subgoal  and  be  more  likely  to  solve  novel  problems  that  involve  that  subgoal  but 
require  different  steps  to  achieve  it. 


What  do  students  learn  from  worked  examples  in  domains 
such  as  probability  and  physics?  Often,  students  memorize 
a  set  of  steps  to  which  they  attach  relatively  little  meaning. 
As  a  result,  when  faced  with  a  test  problem  that  has  the 
same  goal  structure  but  requires  different  steps  for  achiev¬ 
ing  the  goals,  students  may  rigidly  and  incorrectly  apply  the 
old  steps  or  fail  to  produce  a  solution  at  all  (e.g.,  Reed, 
Dempster,  &  Ettinger,  1985;  Ross,  1987,  1989). 

-  Learners  are  more  successful  solving  novel  problems 
^hen  they  learn  the  goal  structure  of  the  problems  in  that 
domain  (e.g.,  Anzai  &  Simon,  1979;  Eylon  &  Reif,  1984). 
Researchers  use  the  term  subgoal  (and  goal)  in  two  differ- 
“«nt  ways.  The  first  defines  a  st^goal  to  be  something 
people — or  computer  programs — form  when  they  are  work¬ 
ing  on  a  problem  and  reach  a  point  where  they  do  not  simply 
recognize  what  to  do  next  because  they  have  no  options,  too 
many  options,  conflicting  options,  etc.  A  subgoal  is  formed 
at  this  impasse  (e.g.,  Newell,  1990,  Chapter  4;  VanLehn, 
1988).  The  second  considers  subgoals  to  represent  the  task 
structure  to  be  learned  for  solving  problems  in  a  particular 
domain  and  assumes  that  these  subgoals  can  be  taught  to 
learners  (e.g.,  Catrambone,  1994a;  Catrambone  &  Holyoak, 
1990;  Dixon,  1987;  Eylon  &  Reif,  1984).  From  the  second 
point  of  view,  a  subgoal  groups  a  set  of  steps  under  a 
meaningful  task  or  purpose  (e.g.,  Anzai  &  Simon,  1979;  Chi 
&  VanLehn,  1991).  For  instance,  in  the  probability  materi¬ 
als  used  in  the  current  experiments,  a  set  of  multiplication 
and  addition  steps  can  be  grouped  under  the  subgoal  “find 
the  total  frequency  of  the  event.”  It  is  the  second,  task- 
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analysis-driven  view  of  subgoals  that  I  foUowed  in  the 
present  study. 

An  instructor  might  help  students  learn  the  goal  structure 
through  worked  examples.  The  particular  subgoals  taught 
might  represent  an  instructor’s  judgment  about  how  stu¬ 
dents  should  decompose  problems  into  subproblems  to 
solve  novel  problems  effectively.  Novel  problems  arc  taken 
to  be  problems  that  share  the  same  goal  or  task  structure 
with  training  examples  but  that  require  a  change  in  the  steps 
for  achieving  at  least  one  of  the  subgoals. 

Much  research  indicates  that  the  way  learners  encode 
examples  influences  how  likely  they  arc  to  access  examples 
and  how  likely  they  arc  to  apply  or  adapt  examples  success¬ 
fully  in  new  situations  (e.g..  Brown,  Kane,  &  Echols,  1986; 
Gentner  &  Gentner,  1983;  Gick  &  Holyoak,  1983).  For 
instance,  Brown  et  al,  found  that  young  children  were  more 
likely  to  use  a  prior  story  to  help  them  solve  an  analogous 
problem  if  the  children  had  either  spontaneously,  or  through 
prompting,  previously  induced  the  goal  structure  of  the 
story. 

My  aim  in  this  study  was  to  examine  whether  learners’ 
transfer  to  novel  problems  is  improved  if  they  study  exam¬ 
ples  designed  to  convey  a  solution  procedure  organized  by 
subgoals  versus  examples  designed  to  convey  only  the  steps 
of  the  solution  procedure. 


Learning  From  Examples 

Learners,  particularly  novices,  typically  prefer  to  study  or 
refer  to  examples,  as  opposed  to  instructions  or  descriptions 
of  principles  of  a  domain,  when  working  on  problems  (e.g., 
Pirolli  &  Anderson,  1985).  Learners  explicitly  mention  ex¬ 
amples  when  solving  a  problem  (Lancaster  &  Kolodner, 
1988;  Ross,  1984),  and  they  follow  examples  rather  than 
instructions  when  the  two  conflict  (LcFcvrc  &  Dixon, 
1986). 

Unfortunately,  people  frequently  do  not  leara  from  exam¬ 
ples  what  is  needed  to  solve  novel  problems.  Rather,  people 
tend  to  memorize  a  set  of  steps.  Attempts  to  improve  this 
situation  have  usually  found  little  improvement  in  transfer. 
For  instance,  Reed  ct  al.  (1985)  provided  elaboration  de¬ 
signed  to  help  students  understand  the  principles  illustrated 
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-  in  algebra  examples.  In  general,  these  elaborations  did  not 
i  -  improve  perfomMuice  on  nonisomorphic  problems.  Similar 
transfer  difficulties  after  learning  from  examples  were  doc- 
'umented  by  Gick  and  Holyoak  (1983),  Ross  (1987,  1989), 
and  others.  However,  several  researchers  have  demonstrated 
some  success  in  using  examples  to  help  learners  transfer  to 
novel  problems  (e.g..  Ward  &  Sweller,  1990;  2^u  &  Simon, 
1987).  It  is  unclear  why  conflicting  transfer  results  are 
"found  aaoss  these  studies.  The  subgoal  framework  offered 
below  may  help  to  identify  situations  in  which  transfer  is 
more  or  less  likely  to  occur. 


Value  of  Learning  Subgoals 

Prior  work  suggests  that  students  can  learn  subgoals  from 
examples.  Chi  and  her  colleagues  (Chi  &  Bassok,  1989; 

*.  Chi,  Bassok,  Lewis,  Reimann,  &  Glaser,  1989;  Chi  & 

"  VanLehn,  1991)  examined  the  self-explanations  lexers 

•  produce  when  studying  examples.  Chi  et  al.  (1989)  divided 
their  students  into  “good  and  poor  students”  (p.  158)  as  a 
function  of  test  problem  performance  that  followed  an 
example-studying  phase.  “Good”  students  demonstrated  su¬ 
perior  transfer  to  novel  problems;  their  self-explanations  of 
the  steps  'from  example  solutions  in  physics  mechanics 

•  problems  contained  more  goals,  descriptions  of  the  precon¬ 
ditions  for  actions,  and  explications  of  the  consequences  of 
actions,  than  did  the  self-explanations  of  “poor”  students 
(Chi  &  VanLehn,  1991). 

Although  students  who  learn  easily  and  students  who 
have  difficulty  learning  may  differ  on  various  dimensions  in 
addition  to  the  self-explanations  they  produce,  one  infer¬ 
ence  that  might  be  drawn  from  Chi  and  her  colleagues’ 
work  is  that  learning  will  be  improved  if  examples  provide 
the  types  of  explanations  that  students  who  learn  easily 
produce  on  their  own.  One  of  these  explanation  types  are 
subgoals. 

Subgoals  are  useful  because  they  group  a  set  of  steps  and 
in  some  sense  explain  what  the  steps  accomplish.  If  the 
learner  can  recognize  which  subgoals  are  relevant  for  solv¬ 
ing  a  novel  problem,  then  those  subgoals  can  guide  the 
learner  to  the  steps  from  the  old  solution  procedure  that 
need  to  be  modified  to  achieve  the  subgoals  in  the  new 
problem.*  In  contrast,  a  learner  who  had  simply  memorized 
a  string  of  steps  for  solving  a  particular  problem  type, 
without  grouping  sets  of  steps  under  the  subgoals  they 
achieve,  will  have  fewer  cues  to  direct  him  or  her  to  the 
steps  that  need  to  be  modified  for  the  novel  probjem. 

A  subgoal  framework  can  be  useful  for  explaining  trans¬ 
fer.  The  framework  can  be  used  to  determine  the  subgoals 
learners  need  to  know  to  solve  problems  in  a  domain,  and  it 
can  be  used  to  guide  the  construction  of  examples  to  in¬ 
crease  the  likelihood  that  learners  learn  these  subgoals. 
Researchers  conducting  studies  within  this  ftamework  can 
examine  issues  such  as  finding  the  most  effective  ways  to 
determine  the  subgoals  in  a  domain  and  finding  effective 
ways  of  conveying  those  subgoals  through  examples. 


Improving  Examples 

In  the  current  study,  a  set  of  steps,  which  was  part  of  the 
overall  solution,  was  labeled  in  training  examples  as  a  way 
of  conveying  the  subgoal  those  steps  achieved.  I  hypothe¬ 
sized  that  example  solutions  that  label  a  set  of  steps  will 
make  a  person  more  likely  to  leam  the  subgoal  achieved  by 
the  steps  rather  than  merely  memorizing  the  steps  them¬ 
selves.  Furthermore,  I  h5q>othesized  that  by  learning  the 
subgoal,  the  learner  will  be  more  likely  to  successfully 
achieve  it  in  a  novel  problem  that  requires  a  different  set  of 
steps  to  achieve  that  subgoal. 

This  second  hypothesis  was  made  for  two  reasons.  First, 
subgoals  were  one  type  of  knowledge  produced  more  often 
by  students  who  learned  easily  than  by  students  who  had 
difficulty  learning  in  Chi  and  her  colleagues’  studies  (e.g., 
Chi  el  al.,  1989).  Second,  subgoal  learning  seems  to  be 
correlated  with  better  performance  at  solving  problems  that 
involve  the  same  subgoals,  even  if  the  problems  are  novel 
and  require  different  steps  to  achieve  the  subgoals  (Anzai  & 
Simon,  1979;  atrambone  &  Holyoak,  1990;  Mawcr  & 
Sweller,  1982). 

The  present  study  continues  a  line  of  research  on  factors 
that  influence  subgoal  learning  (Catrambone,  1994a; 
Catrambone  &  Holyoak,  1990).  Catrambone  and  Holyoak 
(1990)  examined  whether  learners  studying  examples  that 
demonstrated  multiple  ways  of  achieving  a  subgoal  would 
be  more  likely  to  leam  the  subgoal  than  learners  studying 
examples  demonstrating  a  single  method.  The  results  sug¬ 
gested  that  studying  multiple  methods  did  not  aid  subgoal 
learning,  at  least  within  the  confines  of  a  1-hr  e^riment. 
However,  subgoal  learning  did  appear  to  be  aided  when 
learners  studied  examples  that  provided  elaborations  about 
the  particular  methods  used  for  achieving  a  subgoal.  How¬ 
ever,  these  elaborations  contained  a  variety  of  information 
including  additional  domain  theory,  explanations  of  the 
conditions  that  led  to  a  certain  method  being  chosen,  md 
labels  for  key  groups  of  steps.  Thus,  it  is  not  clear  which 
featurc(s)  of  the  elaboration  promoted  subgoal  learning. 

On  the  basis  of  preliminary  research  (Catrambone, 
1994b)  and  the  findings  of  Chi  et  al.  (1989)  suggesting  that 
students  who  leam  easily  tend  to  group  steps  into  subgoals, 
I  examined  whether  a  relatively  minimal  manipulation,  pro¬ 
viding  a  label  for  a  set  of  steps,  would  make  learners  more 
likely  to  form  a  subgoal. 

A  study  by  Smith  and  Goodman  (1984)  supports  the  idea 
that  labels  can  aid  subgoal  learning  and  transfer  to  new 
situations.  The  relevant  comparison  is  between  a  group  of 
students  who  followed  a  set  of  steps  for  assembling  an 
electric  circuit  and  a  group  who  received  a  structurally 
oriented  “explanatory  schema”  (Smith  &  Goodman,  1984, 
p.  360)  with  the  steps.  This  schema  consisted  of  statements 
that  provided  a  rationale  for  why  sets  of  steps  needed  to  be 


*  The  issue  of  learning  to  recognize  when  a  subgoal  is  appro¬ 
priate  for  a  particular  problem  is  not  addressed  here.  Alt  training 
and  test  problems  involved  the  same  subgoals.  Rather,  the  empha¬ 
sis  is  on  learners’  ability  to  achieve  the  subgoal  in  a  new  way  in 
test  problems. 
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carried  out.  Each  rationale  was  essentially  a  statement  of  a 
goal  that  the  steps  achieved  (e.g.,  "The  next  thing  that  you 
will  have  to  do  is  to  assemble  the  on-off  switch.”),  ^en 
assembling  a  new  circuit,  students  who  had  previously 
received  the  explanatory  sdiema  were  more  accurate  m 
building  the  substructures  corresponding  to  the  goals  even 
though  the  required  steps  were  not  identical  to  the  ones 
followed  during  training.  „  j  . 

The  current  study  differs  from  Smith  and  Goodman  s 
(1984)  work  in  several  ways.  First,  during  training  their 
^dents  carried  out  a  set  of  instructions,  whereas  students  in 
the  current  experiments  studied  examples.  Second,  the 
Smith  and  Goodman  transfer  task  involved  the  students’ 
following  a  new  set  of  instructions,  whereas  in  the  present 
study  students  had  to  solve  novel  problems.  Third,  the 
explanatory  sdiema  used  by  Smith  and  Goodman  provided 
a  rationale  for  why  a  set  of  steps  were  carried  out.  Con¬ 
versely,  although  the  labels  used  in  the  present  experiments 
presumably  helped  students  group  a  set  of  steps,  it  was  left 
to  the  students  to  supply  the  rationale  for  why  those  steps 
were  executed.  Thus,  in  the  present  study  I  analyzed  transfer 
in  a  more  demanding  situation  in  which  I  explicitly  exam¬ 
ined  the  role  of  examples  in  subgoal  learning. 

Test  Domain:  The  Poisson  Distribution 

The  Poisson  distribution  is  often  used  to  approximate 
binomial  probabilities  for  events  oocurring  with  some  smaU 
probability.  The  Poisson  equation  is 

[(e-^)(XO] 

P(X  =  x)  = - - , 

where  X  is  the  average  (the  expected  value)  of  the  raiidom 
variable  X,  e  is  the  mathematical  constant  2.718,  and  x  is  the 
number  of  successes  of  interest. 

Consider  an  example  of  the  use  of  the  Poisson  distribution 
in  which  the  average  number  of  briefcases  owned  ^r  law¬ 
yer  is  found  so  that  one  can  predict  the  probability  of  a 
randomly  chosen  lawyer  owning  a  certain  number  of  brief¬ 
cases  (see  Appendix  A).  The  method  for  the  goal  of  finding 
X,  the  average  fi-equency  of  the  event  (e.g.,  owning  a  brief¬ 
case),  could  be  represented  as: 

Goal:  Find  A 
Method: 

1.  Multiply  each  event  category  (e.g.,  owning  ex¬ 
actly  zero  briefcases,  owning  exactly  one  brief¬ 
case,  etc.)  by  its  observed  frequency. 

2.  Sum  the  results. 

3.  Divide  the  sum  by  the  total  number  of  trials  (num¬ 
ber  of  lawyers)  to  find  the  average  number  of 
briefcases  per  lawyer. 

Learners  are  good  at  memorizing,  from  examples,  the 
above  steps  to  adiieve  the  subgoal  of  finding  the  average 
frequency  of  an  event  for  problems  with  different  story¬ 
lines  (atrambone  &  Holyoak,  1990).  However,  they  often 
fail  to  notice  that  Steps  1  and  2  in  the  above  method  could 


also  be  viewed  as  a  method  for  achieving  the  subgoal  of 
finding  the  total  frequency  of  the  event  (e.g.,  total  number  of 
briefcases  owned).  As  a  result,  learners  often  have  trouble 
finding  the  average  frequency  of  an  event  wheri  a  problem 
provides  the  total  frequency  of  that  event  dir^y  (see 
y^pendix  B),  rather  than  requiring  that  it  be  derived  from 
the  frequencies  of  various  event-categories  (Catrambone, 
1994b;  Catrambone  &  Holyoak,  1990). 

If  a  person  learns  the  subgoal  “find  the  total  frequency  of 
the  event,”  he  or  she  might  be  better  able  to  find  X  in  a  novel 
problem  that  requires  a  change  from  the  examples  in  Irow 
total  frequency  is  found.  For  example,  in  the  method  dis¬ 
cussed  earlier  for  finding  the  average  number  of  bnefcases 
owned  per  lawyer,  it  mi^t  be  better  if  the  learner’s  method 
for  finding  X  was  organized  as  follows: 

Goal:  Find  A 
Method: 

1.  Goal:  Find  total  number  of  briefcases. 

Method: 

a.  Multiply  each  event-category  by  its  observed 
frequency. 

b.  Sum  the  results  to  obtain  the  total  number  of 
briefcases. 

2.  Divide  the  total  number  of  briefcases  by  the  total 
number  of  trials  to  obtain  the  average  number  of 
briefcases  per  lawyer. 

Experiment  1 

In  Experiment  1, 1  examined  subgoal  learning  by  manip¬ 
ulating  whether  a  set  of  steps  was  labeled.  It  was  assumed 
that  if  learners  see  a  label  for  a  set  of  steps,  they  are  more 
likely  to  link  those  steps  to  a  common  subgoal.  Subgoal 
learning  was  then  assessed  in  two  ways.  In  the  first,  I 
analyzed  transfer  performance — how  successfully  students 
found  X — on  novel  problems  by  students  who  were  hypoth¬ 
esized  to  have  learned  or  not  to  have  learned  the  subgoal  of 
finding  the  total  frequency  of  an  event.  In  the  second,  I  bad 
these  students  describe  how  to  solve  problems  in  the  Pois¬ 
son  domain.  If  the  descriptions  of  students  in  the  label 
condition  included  statements  such  as  “find  the  total  fre¬ 
quency”  and  if  these  were  the  students  who  also  solved  the 
novel  problems  more  successfully,  then  this  would  provide 
converging  evidence  that  these  students  learned  that 
subgoal. 

In  this  experiment  the  no-Iabel  group  studied  examples 
demonstrating  the  weighted  average  method  for  finding  X 
(see  Appendix  A,  the  no-label  solution).  The  label  group  s 
examples  differed  in  that  the  steps  for  finding  the  to^ 
frequency  were  explicitly  labeled  rather  than  merged  wim 
the  overall  set  of  steps  for  finding  X  (see  Appendix  A,  the 
label  solution). 

In  the  test  phase  students  were  asked  to  (a)  describe  now 
they  would  teach  someone  to  solve  problems  like  the  ones 
they  had  studied  and  (b)  solve  problems  requiring  the  use  of 
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the  weighted  average  method  to  find  A  or  a  method  in  which 
fte  total  frequency  was  supplied  directly  (sec  i^pendix  B). 


Predictions 

It  was  hypothesized  that  the  label  group  would  be  more 
likely  than  the  no-label  group  to  to  learn  the  subgoal  to  find 
the  total  frequency.  As  a  result,  students  in  the  label  group 
should  be  more  likely  than  students  in  the  no-label  group  to 
mention  the  idea  of  finding  a  total  frequency  in  their  de¬ 
scriptions.  It  could  be  argued  that  this  is  a  trivial  prediction 
because  the  label  group  could  simply  be  repeating  a  label  or 
a  generalization  of  a  label  from  the  examples.  Nevertheless, 
if  the  students  who  mention  the  idea  of  finding  total  fre¬ 
quency  are  primarily  those  in  the  label  group  and  if  these 
students  produce  superior  transfer  performance  on  problems 
that  provide  the  total  frequency  directly,  then  support  would 
be  given  to  the  claims  that  labels  aid  subgoal  learning  and 
ghat  subgoals  aid  transfer. 

No  difference  was  predicted  between  the  groups  m  the 
frequency  of  mentioning  the  notion  of  finding  an  average 
(or  A)  in  their  descriptions  because  all  examples  mentioned 

the  term  average  in  the  solutions.  rn  i  * 

Both  groups  were  predicted  to  be  quite  successful  solving 
transfer  problems  that  were  isomorphic  to  the  training  ex> 
amples.  The  label  group  was  expected  to  perform  better  than 
the  no-label  group  on  test  problems  that  provided  the  total 
frequency  directly  ratherlhan  requiring  that  it  be  calculated. 


Method 


Participants.  Participants  were  48  students  recruited  from  an 
Bitroductory  psychology  class  at  the  Georgia  Institute  of  Technol¬ 
ogy  who  received  course  credit  for  their  participation.  None  of  the 
snidents  bad  taken  a  probability  course  before  participating  ui  the 

experiment.  .  , 

Materials  and  procedure.  All  students  initially  studied  a  cover 
sheet  that  briefly  described  the  Poisson  distribution  and  how  it 
could  be  used  as  a  replacement  for  more  cumbersome  techmques 
for  ralrulating  probabilities  involving  events  that  can  be  catego¬ 
rized  as  successes  and  failures.  The  Poisson  equation  was  pre¬ 
sented,  and  a  simplified  notion  of  a  random  variable  was  also 

presented.  ^ 

Students  were  randomly  assigned  to  one  of  two  groups,  ine 
Jabel  group  (A'  =  25)  studied  six  examples  demonstratmg  tte 
weighted  average  method  for  finding  A  in  which  the  steps  for 
finding  the  total  frequency  were  explicitly  labeled  (sw  the  label 
solution  in  Appendix  A  for  an  example).  The  no-label  p^oup  s 
(N  =  23)  examples  differed  in  that  the  steps  for  findmg  the  total 
frequency  were  not  labeled  (see  the  no-label  solution  m  Appendix 
A)  The  labels  seen  by  the  students  in  the  label  group  were  sj^ific 
to  the  context  of  the  problem  (e.g.,  “total  nurnber  of  bnef^ 
owned”)  and  were  not  phrased  at  a  general  level  (e.g.,  total 
•frequency”)  so  that  I  could  minimize  additional  cxpliat  general 

domain  instruction.  . 

After  studying  the  examples,  students  were  ask^  to  describe 
how  to  solve  problems  in  the  domain.  The  instructions  were: 


Suppose  you  were  going  to  teach  someone  how  to  solve 
Poisson  distribution  problems  of  the  types  you  have  just 
studied.  Please  describe  the  procedure  or  procedures  you 


would  give  someone  to  solve  these  problems.  Please  be  as 
complete  as  possible.  Please  do  not  look  back  at  the  examples. 

After  writing  their  descriptions,  students  solved  three  test  prob¬ 
lems  that  required  the  use  of  the  weighted  averse  meth^  for 
finding  A  (isomorphic  to  the  example  in  Appendix  A),  and  then 
they  solved  three  test  problems  in  which  total  frequency  was  given 
directly  (and  thus  A  could  be  found  by  simply  dividing  *c  pven 
number  by  the  total  number  of  trials).  The  latter  type  mcluded  Ae 
problem  in  Appendix  B  and  two  problems  isomorphic  to  it.  in¬ 
dents  were  told  not  to  look  back  at  the  examples  when  solvmg  the 

test  problems.  ^  r  ^ 

Students’  written  solutions  were  scored  for  whether  they  found 
A  correctly.  In  addition,  students’  descriptions  of  how  to  solve 
problems  in  the  domain  were  scored  for  two  primary  features:  an 
explicit  mention  of  trying  to  find  the  total  frequency  and  an 
explicit  mention  of  trying  to  find  an  average.  Two  raters  mde- 
pcndently  scored  the  descriptions  and  problem  solutions  and 
agreed  on  scoring  94%  of  the  time.  Disagreements  were  resolved 
by  discussion. 


Results 


Students  were  given  a  score  of  1  for  a  given  problem  if 
they  found  A  correctly  and  a  score  of  0  otherwise.  The 
scores  for  the  three  problems  that  were  isomorphic  to  the 
training  examples.  Problems  1-3,  were  summed,  thus  giv¬ 
ing  students  a  score  ranging  from  0  to  3  for  their  perfor¬ 
mance  on  those  problems.  SimUarly,  the  scores  for  the  three 
novel  problems.  Problems  4-6,  were  summ^,  thus  givmg 
students  a  score  ranging  from  0  to  3  for  their  performance 


on  those  problems.  j  u 

Transfer  as  a  function  of  group.  As  expected,  both 
groups  did  quite  well  at  finding  A  on  test  problems  that  were 
isomorphs  to  the  training  examples  (Problems  1-3),  Md 
there  was  no  significant  difference  in  performance,  r(l, 
46)  =  0.28,  p  =  .60,  MSB  =  .40.  The  average  for  the  label 
group  was  2.9,  and  the  average  for  the  no-label  group 
2.S. 

As  predicted,  however,  the  label  group  found  A  more 
successfully  than  did  the  no-label  group  on  the  test 
nroblems  that  involved  the  new  method  for  finding  A  (Prob¬ 
lems  4-6),  F(l,  46)  =  5.32,  p  =  .03,  MSB  =  1.82.  The 
average  for  the  label  group  was  2.2,  and  the  average  for  the 
no-label  group  was  1.3.  The  most  frequent  mstake  that 
students  made  on  these  problems  was  to  w^te  in  the  solu¬ 
tion  area  that  not  enough  information  was  given  to  solve  the 

problem.  „  ,  , 

Descriptions  as  a  function  of  group.  Students  desenp- 
tions  were  scored  for  whether  they  explicitly  mentioned 
finding  the  total  frequency  and  the  subgoal  of  finding  the 
average  or  A.  As  expected,  the  groups  were  equally  likely  to 
mention  finding  the  average  (label,  60%  and  no-label, 
65%),  x^(l)  =  0.14,  p  =  .71.  As  predicted,  the  label  ^oup 
mentioned  finding  total  frequency  more  often  than  ^d  the 
no-label  group  (52%  vs.  13%),  x  (1)  “  P  ~ 

Transfer  as  a  function  of  descriptions.  Students  who 
mentioned  finding  the  total  frequency  in  their  descriptions 
(N=  16)  tended  to  perform  about  the  same  on  the  isomor¬ 
phic  test  problems  when  compared  with 
not  mention  it  (N  =  32),  f(l,  46)  =  1.70,  p  =  .20,  MSB  - 
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0.39  (Ms  =  3.0  and  2.75,  respectively).  As  predicted,  stu¬ 
dents  who  mentioned  finding  the  total  frequency  did  better 
than  did  the  other  students  at  finding  A  on  test  problems  that 
required  the  new  method  for  finding  A,  F(l,  46)  ==  5.52, p  — 
.02,  MSE  -  1.81  (Ms  =  2J58  and  1.41,  respectively). 

Students  who  mentioned  finding  the  average  in  then- 
descriptions  {N  =  30)  did  not  perform  significantly  differ¬ 
ently  on  the  isomorphic  test  pr^lems  when  compared  with 
students  who  did  not  mention  it  {N  =  18),  F(l,  46)  —  2.06, 
p  =  .16,  MSE  =  0.39,  (Ms  =  2.93  and  2.67,  respectively). 
There  was  also  no  significant  difference  between  these 
groups  at  finding  A  for  the  problems  that  required  the  new 
method  for  finding  A,  F(l,  46)  =  0.20,  p  -  .66,  MSE  =  2.02 
(Ms  =  1.80  and  1.61,  respectively). 


Discussion 

Students  in  the  no-label  group  were  not  able  to  find  A  as 
successfully  in  the  novel  problems  (Problems  4-6)  as  were 
the  students  in  the  label  group.  This  result  suggests  that 
students  in  the  label  group  were  more  likely  to  learn  the 
subgoal  of  finding  the  total  frequency  as  part  of  the  solution 
-structure  for  solving  Poisson  distribution  problems.  As  a 
-result,  the  students  in  the  label  group  were  able  to  find  A  in 
problems  that  could  not  be  solved  with  the  steps  from  the 
training  examples. 

Converging  evidence  that  the  students  in  the  label  group 

had  learned  the  subgoal  to  find  the  total  frequency  comes 
from  students’  descriptions  of  how  to  solve  problems.  Stu- 
-  dents  in  the  label  group  were  four  times  more  likely  than 
students  in  the  no-label  group  to  mention  the  goal  of  finding 
the  total  frequency.  Both  groups  mentioned  the  goal  of 
finding  A  equally  often  which  was  consistent  with  the  fact 
that  A  was  labeled  in  the  training  examples  for  both  groups. 

It  could  be  argued  that  those  students  who  wrote  descrip¬ 
tions  in  which  they  mentioned  the  subgoal  of  finding  the 
total  frequency  were  also  people  who  were  simply  better  at 
transfer.  However,  the  finding  that  the  label  and  no-label 
groups  had  differential  far  transfer  success  (i.e.,  success  at 
solving  problems  requiring  new  or  modified  steps  compared 
to  the  examples)  suggests  that  the  training  manipulation 
affected  subgoal  learning  in  addition  to  any  effects  that  were 
due  to  individual  differences. 

The  results  from  this  experiment  are  consistent  with  the 
claim  that  manipulations  to  examples  that  promote  attention 
to  subgoals  can  help  students  learn  those  subgoals.  Students 
in  the  label  group  learned  the  subgoal  to  find  the  total 
frequency,  and  this  subgoal  helped  their  performance  on  test 
problems  that  required  a  new  way  to  find  the  total  fie- 
qucncy.  However,  this  new  way  was  to  recognize  that  the 
total  frequency  was  given  in  the  problem.  Perhaps  a  more 
stringent  test  of  whether  learning  a  subgoal  helps  a  learner 
to  achieve  it  when  a  novel  method  is  required  would  be  to 
give  students  test  problems  that  require  a  new  method  of 
calail^ting  total  frequency.  Experiment  2  was  designed  to 
do  this. 


Experiment  2 

In  Experiment  2, 1  examined  students’  ability  to  modify 
an  old  method  for  calculating  the  total  frequency  to  find  A. 
As  in  Experiment  1,  the  steps  for  finding  the  total  frequency 
were  either  labeled  or  unlabeled  in  the  training  examples. 

The  new  method  for  finding  total  frequency  was  to  add 
the  frequencies  of  a  number  of  events  rather  than  to  multiply 
event-  categories  by  their  frequeipes  and 
products  (such  as  the  method  used  in  .^)pendix  A).  One  of 
the  two  test  problems  requiring  this  modified  method  re¬ 
volved  children  finding  seashells  on  the  beach  (see  Appen¬ 
dix  C).  The  numbers  of  shells  found  by  a  child  on  Day  1, 
Day  2,  Day  3,  Day  4,  and  Day  5  are  given.  Hie  problem 
then  asks  for  the  probability  of  a  randomly  chosen  duld 
finding  a  certain  number  of  shells.  If  the  label  group  is  inore 
likely  than  the  no-label  group  to  learn  the  subgoal  of  findmg 
the  total  frequency,  then  the  students  in  the  label 
should  be  more  successful  than  the  students  in  the  no-lawl 
group  at  modifying  their  approach  for  finding  total  fre¬ 
quency  and  thus,  finding  A  in  this  problem. 

The  solutions  presented  for  the  training  examples  were 
the  same  as  in  Experiment  1  except  that  the  steps  for  findmg 
total  frequency  and  finding  A  were  circled,  either  separately 
or  together  (see  Figure  1).  Students  saw  both  presenteUoiK 
for  each  example  and  were  asked  to  pick  the  one  they  relt 
circled  the  steps  that  “go  together.”  It  was  hypothesized  that 
if  a  student  learned  the  subgoal  to  find  the  total  frequency, 
then  he  or  she  would  be  more  likely  to  prefer  that  the  steps 
for  finding  the  total  frequency  be  separated  from  the  step  for 
finding  A.  (This  is  the  solution  in  Figure  lA  for  the  label 
students  and  Figure  1C  for  the  no-label  students.) 

For  each  test  problem,  students  were  asked  to  circle  the 
parts  of  their  solutions  that  went  together.  Again, 
students  in  the  label  group  were  predicted  to  be  more  likely 
than  students  in  the  no-label  group  to  have  le^ed  the 
subgoal  to  find  the  total  frequency,  it  was  predicted  toat 
students  in  the  label  group  would  be  more  likely  Aan 
students  in  the  no-label  group  to  circle  the  steps  for  fedmg 
the  total  frequency  separately  from  the  step  for  finding^  A. 
Students’  circling  performance  could  provide  convergmg 
evidence,  along  with  their  transfer  performance,  for  subgoal 
learning. 


Method 

Panidpants.  Participants  were  52  students  from  an  introduc¬ 
tory  psychology  class  at  the  Georgia  Institute  of  Technology  who 
participated  for  course  aedit.  None  of  the  students  had  taken  a 
probability  course  before  participating  in  the  experiment  Smdents 
were  randomly  assigned  to  the  label  {N  =  26)  and  no*labcl  {N 
26)  grout)s. 

Mcaeriats  and  procedure.  Both  groups  studied  the  same  cover 
sheet  used  in  Experiment  1  and  then  studied  three. exarnpl^ 
illustrating  the  weighed  average  method  of  finding  A.  For  the  label 
KToup,  the  subgoal  of  finding  the  total  frequency  of  the  event  ^ 
labeled.  This  subgoal  was  not  labeled  for  the  no-label  group.  The 
examples  were  a  subset  of  those  used  in  Experiment  1. 

Students  saw  two  solutions  to  each  example.  The  soluUons  were 
identical  except  for  how  the  steps  were  circled.  For  instance,  for 
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A.  Label  Students,  Circled  Separately 


B.  Label  Students,  Circled  Together 


C.  No-Label  Students.  Circled  Separately 


D.  No-Label  Students,  Circled  Together 


Figure  I  Example  solutions  for  the  problem  in  Appendix  A  in  which  the  steps  for  finding  the  total 
^uency  and  X  It  circled  either  separately  or  together  m  Experiment  2. 
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the  training  example  in  Appendix  A,  students  in  the  label  poup 
saw  the  solutions  in  Figures  lA  and  IB,  whereas  students  in  the 
no-label  group  saw  the  solutions  in  Figures  1C  and  ID.  Students 
were  given  the  following  instructions: 

For  each  example,  please  note  that  the  solution  is  presented 
twice.  The  two  presentations  of  each  solution  have  the  steps 
circled.  The  circles  are  used  to  indicate  steps  that  might  “go 
together.”  For  instance,  suppose  you  were  following  a  recipe 
for  cooking  something.  Perhaps  the  first  three  steps  of  die 
recipe  involved  putting  various  ingredients  into  a  bowl  and  the 
fourth  step  involved  stirring  the  ingredients  with  a  spoon  and 
the  fifth  step  involved  using  a  blender  to  finishing  the  mixing. 
You  might  draw  a  circle  around  the  first  three  steps  because 
they  involve  “adding  ingredients,”  and  you  mi^t  draw  a 
circle  around  the  the  fourth  and  fifth  steps  because  they 
involve  “blending  the  ingredients.”  The  two  solutions  for  each 
example  are  identical  except  that  difierent  steps  are  in  the 
circles.  For  each  example,  indicate  whether  Presentation  1  or 
Presentation  2  is  the  one  with  which  you  most  agree. 

As  a  counterbalancing  measure,  the  solutions  that  had  the  steps 
for  finding  the  total  frequency  and  A  in  the  same  circle  came  first 
for  half  of  the  students  in  each  group,  whereas  for  the  rest  of 
the  students  the  solution  that  had  the  steps  for  finding  the  total 
frequency  and  finding  A  circled  separately  came  first.  The  solu¬ 
tion  order  had  no  effect  on  students’  preferences  for  solutions  or 
^n  transfer  performance,  and  thus,  was  collapsed  over  for  all 
;-analyses. 

After  studying  the  examples,  students  solved  five  target  prob- 
-^ems.  The  first  was  a  weighted  average  problem  isomorphic  to  the 
examples.  The  second  also  involved  finding  A  as  a  weighted 
‘average;  however,  the  divisor  (total  number  of  trials)  was  not  given 
directly  in  the  problem.  Rather,  it  had  to  be  found  by  adding  the 
number  of  members  in  each  category  (see  Appendix  D).  The  third 
«  provided  the  value  of  the  total  frequency  directly  (see  Appendix 
B).  The  fourth  and  fifth  involved  a  modification  of  the  old  method 
for  finding  the  total  frequency:  instead  of  the  total  frequency  being 
found  by  multiplying  the  event-categories  by  their  frequencies  and 
then  summing,  it  was  found  by  adding  a  set  of  frequencies  (see 
y^pendix  C  for  an  example). 

After  they  solved  each  test  problem  students  were  asked  to 
circle  the  steps  of  their  solution  that  went  together.  Students  were 
told  not  to  look  back  at  the  examples  when  working  on  the  test 
problems. 

Predictions 

Training  examples  solution  preferences.  Students  in  the  label 
group  were  predicted  to  be  more  likely  than  students  in  the 
no-label  group  to  prefer  solutions  in  which  the  steps  for  finding  the 
total  frequency  were  circled  separately  from  the  steps  for  finding 
A.  However,  it  should  be  noted  that  the  “circled  separately”  pre¬ 
sentation  for  the  students  in  the  no-label  group  (sec  Figure  IQ 
looks  a  bit  odd  because  a  set  of  steps  that  arc  part  of  a  fraction  arc 
cirded,  and  the  denominator  is  left  out  of  the  circle.  Thus,  students 
could  be  predisposed  not  to  choose  this  presentation. 

Tranter  performance.  It  was  predicted  that  both  groups  would 
do  well  on  the  first  test  problem,  an  isomorph  to  the  training 
examples.  For  the  second  problem  both  groups  were  expected  to 
perform  similarly.  Given  that  the  subgoal  of  finding  the  number  of 
trials  was  not  emphasized  in  the  training  examples  for  cither  group, 
performance  was  expected  to  be  poor.  As  in  Experiment  1,  the 
label  group  was  predicted  to  do  better  at  finding  A  on  the  third  test 
problem,  which  involved  the  recognition  that  the  total  frequency 


was  given  directly.  The  label  group  was  predicted  to  do  better  on 
the  fourth  and  fifth  test  problems,  which  involved  finding  the  total 
frequency  by  adding  a  set  of  frequencies.  The  subgoal  of  finding 
the  total  frequency  should  have  aided  students  in  the  label  group  in 
figuring  out  which  steps  they  needed  to  modify  to  find  the  total 
frequency. 

^gmenting  of  solutions  to  test  problems.  Students  in  the  label 
group  were  predicted  to  be  more  likely  than  students  in  the 
no-label  group  to  circle  the  steps  for  finding  the  total  frequency 
separately  from  the  step  for  finding  A  in  their  solutions  to  the  test 
problems  that  involve  calculating  the  total  frequency  (Problems  1, 
2,  4,  and  5).  No  circling  difference  was  predicted  between  the 
groups  for  the  problem  in  which  the  total  frequency  was  given 
directly  (Problem  3)  because  it  would  have  b^n  rather  odd  to 
circle  the  given  total  frequency  as  a  step. 

Two  raters  independently  scored  the  example  solution  prefer¬ 
ences  and  problem  solutions,  and  they  agreed  88%  of  the  time. 
Disagreements  were  resolved  by  discussion. 


Results  and  Discussion 

Training  examples  solution  preferences.  A  given  stu¬ 
dent  invariably  chose  the  same  circling  scheme  across  the 
three  examples  (i.e.,  cither  the  one  in  which  the  steps  for 
finding  total  frequency  and  the  step  for  finding  A  were 
circled  separately  or  the  one  in  which  they  were  circled 
together).  This  was  not  surprising  because  Ac  training  ex¬ 
amples  were  isomorphs.  Thus,  students  were  categorized 
into  one  of  two  groups:  those  who  preferred  separate  circles 
for  total  frequency  and  A  and  Aosc  who  preferred  Aat  total 
frequency  and  A  be  in  the  same  circle.  As  predicted,  stu¬ 
dents  in  the  label  group  were  more  likely  Aan  students  in 
the  no-label  group  to  choose  solutions  in  which  the  steps  for 
finding  the  total  frequency  were  circled  separately  from  the 
step  for  finding  A  (50%  vs,  19%),  >^(1)  =  5.44,  p  =  .02. 

Transfer  performance.  All  but  one  student,  from  the 
label  group,  correctly  solved  the  first  test  problem,  a 
weighted  average  problem  isomorphic  to  the  training 
examples. 

The  second  test  problem  was  also  a  weighted  average 
problem,  but  Ae  total  number  of  trials,  Ae  value  Aat  would 
be  placed  in  the  denominator  when  calculating  A,  was  not 
given  directly.  Rather,  the  student  had  to  calculate  Ac  total 
number  of  trials  by  adding  Ac  number  of  workers.  As 
expected,  the  proportion  of  students  who  found  A  correctly 
in  each  group  did  not  differ  significantly  (label,  85%  and  no 
label,  73%),  ;^(1)  =  1.04,  p  =  .31,  although  the  overall 
performance  was  higher  than  expected. 

The  third  test  problem  provided  the  value  for  Ae  total 
frequency  directly.  As  predicted,  the  label  group  correctly 
found  A  significantly  more  often  Aan  did  Ac  no-label  group 
(88%  vs.  54%),  x^(l)  =  7.59,  p  =  .006.  This  replicates  Ac 
finding  from  &q>erimcnt  1. 

The  fourth  and  fifth  test  problems  required  Ac  s^dents  to 
calculate  Ae  total  frequency  by  adding  Ac  simple  frequen¬ 
cies  (c.g.,  Ae  number  of  shells  found  each  day).  Students 
were  given  a  score  of  1  for  a  given  problem  if  Aey  found  A 
conectly  and  a  score  of  0  oAcrwise.  The  scores  for  Ae  two 
problems  were  summed,  thus  giving  students  a  score  rang¬ 
ing  from  0  to  2  for  their  performance  on  Aose  problems. 
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As  predicted,  the  label  group  found  A  more  successfully 
than  did  the  no-labcl  group,  F(l,  50)  =  4.15,  p  =  .047, 
MSE  =  0.46  {Ms  =  1.50  and  1.12,  respectively).  Students 
who  solved  these  problems  incorrectly  tended,  for  example, 
to  multiply  day  by  number  of  shells  (Problem  4)  or  wnte 
that  not  enough  information  was  given.  This  result  suggests 
that  learning  a  subgoal  does  aid  a  person  in  achieving  it  in 
a  novel  problem  that  requires  a  modification  of  an  old 
method. 

Segmenting  of  solutions  to  test  problems.  For  each  test 
problem  that  required  A  to  be  calculated  problems  1,  2, 4, 
and  5),  students  were  given  a  score  of  1  if  they  circled  the 
steps  for  finding  the  total  frequency  separately  from  the  step 
for  fmding  A  in  their  solution  and  a  score  of  0  otherwise. 
The  scores  for  the  problems  were  summed,  thus  giving 
students  a  score  ranging  from  0  to  4. 

As  expected,  students  in  the  label  group  were  more  likely 
to  circle  the  steps  for  finding  the  total  frequency  separately 
from  the  step  for  finding  A  in  their  solutions  than  were  the 
sfcdents  in  the  no-label  group,  F(l,  50)  =  8.39,  p  =  .006, 
j05£  =  1.32  {Ms  =  1.15  and  0.23,  respectively). 

Also  as  expected,  there  was  no  difference  between  the 
groups  on  Problem  3  in  which  the  total  frequently  was 
provided  directly,  ;t^(l)  =  2.08,  p  =  .15.  Only  two  students, 
both  in  the  label  group,  circled  the  total  frequency.  The  rest 
of  the  students  simply  circled  the  entire  set  of  calculations 
that  they  used  for  finding  A  (if  they  found  a  value  for  A). 


Experiment  3 

The  results  from  the  first  two  experiments  are  consistent 
with  the  interpretation  that  the  design  of  examples  that 
promote  attention  to  subgoals  can  be  successful  in  helping 
students  to  learn  those  subgoals  and  to  transfer  more  suc¬ 
cessfully.  In  both  experiments  a  labeling  manipulation  was 
used  to  promote  attention  to  the  subgoals.  However,  the 
examples  containing  a  label  for  total  frequency  differed 
from  the  no-label  examples  in  an  additional  way:  The  steps 
for  finding  total  frequency  were  on  a  separate  line  from  the 
rest  of  the  steps  for  finding  A.  Thus,  it  is  possible  that  this 
visual  isolation  either  by  itself  or  in  combination  with  the 
label,  produced  the  superior  transfer.  Perhaps  mampulations 
that  lead  learners  to  group  a  set  of  steps  will  make  those 
teamers  more  likely  to  form  a  subgoal  to  relate  those  steps, 
in  Experiment  3,  I  examined  this  possibility  with  four 
groups  of  students. 

The  visual-isolation  group  studied  example  solutions  such 
as  the  one  in  Appendix  E  (this  is  a  solution  to  the  problem 
in  Appendix  A)  in  which  the  total  frequency  steps  were  on 
their  own  line  without  a  label.  The  separate-line  label  group 
studied  example  solutions  isomorphic  to  those  seen  by  the 
label  group  in  Experiment  1  (see  Appendix  A).  Note, 
though,  that  this  solution  style,  besides  labeling  the  steps  for 
finding  the  total  frequency,  places  those  steps  on  a  separate 
line  from  the  rest  of  the  steps  for  finding  A.  Thus,  students 
in  this  group  saw  solutions  that  involved  both  labeling  and 
visual  isolation.  To  examine  whether  there  was  any  inter¬ 
action  between  labeling  and  visual  isolation,  I  had  the 


same-line  label  group  study  examples  that  had  the  steps  for 
finding  the  total  frequency  labeled  but  had  those  steps 
located  on  the  same  line  as  the  rest  of  the  steps  for  finding 
A  (see  Appendix  E).  Finally,  a  no-label  group  analogous  to 
the  group  from  Experiment  1  (see  ^pendix  A)  was  in¬ 
cluded  as  a  baseline. 

According  to  the  label  view,  the  label  is  primarily  respon¬ 
sible  for  subgoal  learning.  Thus,  students  who  study  exam¬ 
ples  in  which  the  steps  for  finding  total  frequency  are 
labeled — the  separate-line  label  and  the  same-line  label 
groups— should  outperform  the  visual-isolation  and  no- 
label  groups  on  transfer  problems  because  the  steps  for 
finding  total  frequency  were  not  labeled  in  the  examples 
studied  by  the  latter  two  groups. 

According  to  the  grouping  view,  grouping-— and  thus 
subgoal  learning — ^is  promoted  by  labeling  or  visual  isola¬ 
tion.  Under  this  view,  the  label  groups  and  the  visual- 
isolation  group  should  perform  similarly,  and  all  these 
groups  should  outperform  the  no-label  group.  Neither  view 
explicitly  predicted  an  interaction  between  labelmg  and 
visual  isolation.  However,  if  a  label  and  visual  isolation  are 
required  for  grouping  and  subgoal  formation,  then  the 
separate-line  label  group  should  perform  better  than  all 
other  groups  on  the  transfer  problems. 


Method 

Participants.  Participants  were  118  students  from  an  introduc¬ 
tory  psychology  class  at  the  Georgia  Institute  of  Technology  who 
participated  for  course  credit.  None  of  the  studente  had  taken  a 
probability  course  before  participating  in  the  experiment. 

Materials  and  procedure.  All  students  studied  the  same  cover 
sheet  used  in  the  prior  experiments  and  then  studied  two  isomor¬ 
phic  examples  illustrating  the  weighted  average  method  for  finding 
A.  The  examples  were  a  subset  of  those  used  in  Experiment  1. 
Fewer  study  examples  were  used  in  this  experiment  because  pilot 
testing  with  students  receiving  six,  three,  or  two  examples  showed 
no  effect  of  number  of  examples  on  performance. 

Students  were  randomly  assigned  to  one  of  four  groups.  Students 
in  the  same-line  label  group  {N  =  30)  had  the  steps  for  finding  the 
total  frequency  labeled,  but  the  steps  were  on  the  same  line  as  the 
rest  of  the  steps  for  finding  A.  The  solution  to  the  problem  in 
Appendix  A  studied  by  the  same-line  label  group  is  shown  in 
Appendix  E.  The  scparate-line-label  group  {N  =  30)  also  had  the 
steps  for  finding  total  frequency  labeled,  but  they  were  on  a 
separate  line  from  the  rest  of  the  steps  for  finding  A.  The  solution 
to  the  problem  in  Appendix  A  that  was  studied  by  the  separate-line 
label  group  was  identical  to  the  one  studied  by  the  label  group  in 
Experiment  1  (see  Appendix  A).  The  visual-isolation  group  (N  = 
29)  studied  examples  in  which  the  steps  for  finding  total  frequency 
were  unlabeled  but  on  their  own  line  (see  Appendix  E).  The 
no-label  group  {N  =  29)  saw  solution  types  identical  to  those 
studied  by  the  no-label  group  in  Experiment  1  (see  Appendix  A). 

As  in  Experiment  1,  students  were  asked  to  describe  how  to 
solve  problems  in  the  domain  after  they  finished  studying  the 
examples.  After  writing  their  descriptions,  students  solved  three 
test  problems.  The  first  problem  required  the  use  of  the  weight^ 
average  method  for  finding  A  (isomorphic  to  the  example  in 
Appendix  A).  The  second  problem  provided  the  total  frequency 
directly  (see  Appendix  B).  The  third  problem  involved  the  addition 
of  simple  frequencies  to  get  the  total  frequency  (see  Appendix  Q. 
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Students  were  told  not  to  look  back  at  the  examples  when  solving 

the  test  problems.  .  .  ,  j 

Students’  written  solutions  were  scored  for  whether  they  found 
A  correctly.  Students’  descriptions  of  how  to  solve  problems  in  the 
domain  were  scored  for  whether  the  students  mentioned  trying  to 
And  the  total  frequency  and  trying  to  find  an  average.  Two  raters 
independently  scored  the  descriptions  and  problem  solutions  and 
agreed  on  scoring  90%  of  the  time.  Disagreements  were  resolved 
by  discussion. 


Results  and  Discussion 

Tranter  as  a  function  of  group.  As  expected,  all  groups 
did  quite  well  at  finding  A  on  the  test  problem  that  was 
isomorphic  to  the  training  examples.  Only  two  students,  one 
in  the  same-line  label  group  and  one  in  the  separate-line 
label  group,  solved  this  problem  incorrectly. 

For  the  two  far  transfer  problems,  students  were  given  a 
score  of  1  for  a  given  problem  if  they  found  A  conectly  and 
a  score  of  0  otherwise.  The  scores  for  the  two  problems 
were  summed,  thus  giving  students  a  score  ranging  from  0 
to  2  for  their  performance  on  those  problems. 

There  were  significant  differences  among  the  four  groups 
with  respect  to  finding  A  in  the  novel  test  problems,  F(3, 
114)  =  5.53,  p  =  .0014,  MSB  =  0.52,  with  means  of  1.7, 

<  1.6,  1.5,  and  1.0  for  same-line  label,  separate-line  label, 
visual-isolation,  and  no-label  groups,  respectively.  As  pre¬ 
dicted  by  the  grouping  view,  Shaffer  (1986)  sequential 
Bonferroni  pairwise  comparisons  (familywise  a  =  .05) 
indicated  that  both  label  groups  and  the  visual-isolation 
group  outperformed  the  no-label  group,  all  ps  <  .0167.  Also 
consistent  with  the  grouping  view,  but  not  with  the  label 
view,  was  the  finding  that  no  reliable  performance  differ¬ 
ences  were  found  between  the  two  label  groups  or  between 
either  label  group  and  the  visual-isolation  group  (all 
ps  >  .05). 

Descriptions  as  a  function  of  group.  Students  descrip¬ 
tions  were  scored  for  whether  they  explicitly  mentioned  the 
subgoal  of  finding  the  total  frequency  and  the  subgoal  of 
finding  the  average  or  A.  As  expected,  the  groups  were 
equally  likely  to  mention  finding  A  (same-line  label,  87%, 
separate-line  label,  87%;  visual  isolation,  76%;  and  no 
label,  83%),  x^(3)  =  1.62,  p  =  .65. 

There  were  significant  differences  among  the  four  groups 
with  respect  to  the  frequency  of  mentioning  finding  total 
frequency,  x^(3)  =  17.43,  p  =  .0006,  with  percentages  of 
43%,  57%,  31%,  and  7%  for  same-line  label,  separate-line 
label,  visual-isolation,  and  no-label  groups,  respectively. 
Consistent  with  the  grouping  view  and  the  tr^sfer  results, 
pairwise  comparisons  (familywise  a  =  .05)  indicated  that 
the  two  label  groups  each  mentioned  finding  total  frequency 
more  often  than  did  the  no-label  group  (both  ps  <  .0167,  as 
did  the  visual-isolation  group  (p  <  .0167,  one-tailed).  Dif¬ 
ferences  among  the  separate-line,  same-line,  and  visual- 
isolation  conditions  were  not  statistically  significant  accord¬ 
ing  to  the  Shaffer  (1986)  sequential  Bonferroni  procedure, 
aU  ps  >  .025. 

Transfer  as  a  function  of  descriptions.  As  expected, 
there  were  no  significant  performance  differences  on  the 


novel  test  problems  between  students  who  mentioned  find¬ 
ing  A  (A^  =  98)  and  those  who  did  not  {N  =  20),  F(l, 
116)  =  1.04,  p  =  .31,  MSB  =  0.57  (Afs  =  1.5  and  1.3, 
respectively). 

Students  who  mentioned  finding  the  total  frequency  (N  - 
41)  were  more  successful  than  other  participants  {N  =  77) 
at  finding  A  on  the  novel  test  problems,  f(l,  116)  =  5.77, 
p  =  .018,  MSB  =  .55  (Ms  =  1.7  and  1.3,  respectively). 

The  results  of  Experiment  3— similar  transfer  perfor¬ 
mance  by  the  label  groups  and  visual-isolation  groups,  and 
better  performance  than  the  no-label  group — support  the 
grouping  view  of  subgoal  formation  and  also  suggest  that 
there  is  no  apparent  interaction  between  labeling  and  visual 
isolation  on  subgoal  formation.  The  tendency  of  the  label 
and  visual-isolation  groups  to  mention  finding  total  fre¬ 
quency  in  their  descriptions  more  often  than  the  no-label 
group  is  also  consistent  with  the  claim  that  these  students 
were  more  likely  to  form  a  subgoal  for  finding  total 
frequency. 

Finally,  students  who  mentioned  finding  total  frequency 
in  their  descriptions  showed  better  transfer  on  the  novel 
transfer  problems.  Students  in  Experiment  1  who  mentioned 
total  frequency  also  transferred  better  on  the  novel  prob¬ 
lems.  In  addition,  in  a  prior  study  (Catramlwne,  1994b)  1 
also  found  this  relationship  between  descriptions  and  trans¬ 
fer  performance.  Together,  these  studies  suggest  that  stu¬ 
dents’  descriptions  provide  a  potentially  useful  measure  of 
subgoal  learning.  The  transfer  and  description  results  pro¬ 
vide  converging  evidence  for  the  hypothesis  that  manipula¬ 
tions  that  encourage  grouping  enhance  subgoal  learning  and 
thus,  transfer. 

General  Discussion 

Learners’  problem-solving  knowledge  in  domains  such  as 
probability  often  seems  to  be  focused  on  the  mathematical 
steps  illustrated  in  training  examples.  This  translates  into 
poor  performance  on  transfer  problems.  If  subgoals  could  be 
conveyed  to  learners,  then  learners  should  be  more  succe^- 
ful  on  novel  problems.  One  way  these  subgoals  can  md 
learners  is  by  guiding  them  to  the  steps  in  the  old  solution 
procedure  that  need  to  be  changed  to  achieve  the  subgoals  in 
a  novel  problem. 

All  three  experiments  presented  here  suggest  that  if  p^ 
pie  are  led  to  learn  a  subgoal,  in  this  case  either  by  studying 
examples  that  label  the  steps  that  achieve  the  subgo^  or  by 
visually  isolating  those  steps,  then  they  are  more  likely  to 
successfully  achieve  it  in  a  novel  problem.  It  is  impressive 
to  find  a  performance  difference  between  the  label  and 
visual-isolation  groups  compared  with  the  no-label  groups 
given  that  the  new  methods  for  finding  the  total  frequency 
in  the  test  problems  were  seemingly  straightforward:  adding 
a  set  of  frequencies  or  recognizing  that  the  total  frequency 
was  given  directly.  This  may  partly  be  an  effect  that  is  due 
to  Binstellung  or  mechanization  (Luchins,  1942):  Students 
may  have  been  biased  against  altering  a  memorized  set  of 

*  Besides  transfer  differences,  the  experiments  provide  ad¬ 
ditional  evidence  that  manipulations  that  encourage  group- 
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ing  can  help  people  learn  subgoals.  Across  Experiments  1 
and  3,  descriptions  given  by  students  in  the  label  and 
visual'isolation  groups  of  how  to  solve  Poisson  problems 
mentioned  the  goal  to  End  the  total  frequency  more  often 
than  did  descriptions  given  by  students  in  the  no-label 
group.  In  Experiment  2,  students  in  the  label  group  pre- 
fened,  more  than  did  students  in  the  no-label  group,  exam¬ 
ple  solutions  that  separated  the  steps  for  finding  total  fre¬ 
quency  from  the  step  for  finding  A.  Students  in  the  label 
group  were  also  more  likely  than  were  students  in  the 
no-label  group  to  circle  the  total  frequency  steps  as  a  unit  in 
their  solutions  to  the  test  problems. 

The  superior  transfer  performance  of  the  label  and  visual- 
isolation  groups  across  the  experiments  is  particularly  im¬ 
pressive  in  light  of  the  extensive  elaboration  provided  to 
students  in  the  study  by  Catrambone  and  Holyoak  (1990).  In 
that  study,  the  overall  transfer  advantage  for  students  re¬ 
ceiving  elaboration,  although  not  reliable  across  all  far 
transfer  problems,  was  in  line  with  performance  by  students 
inithe  label  and  visual-isolation  groups  in  the  present  study 
who  received  a  much  more  minimal  manipulation.  Thus,  it 
appears  that  the  label  and  visual  isolation  manipulations 
may  be  distillations  of  the  information  that  provided  the 
most  benefit  in  the  earlier  study. 

Related  problem  solving  research  by  Reed  et  al.  (1985) 
and  Ross  (1987,  1989)  demonstrates  that  learners  become 
attached  to  superficial  details  and  mathematical  procedures 
of  examples  in  lieu  of  acquiring  more  generalized  knowl- 
'Cdge  about  how  to  solve  problems  in  a  particular  domain. 
Although  Reed  et  al.  attempted  to  provide  elaborations  to 
help  students  go  beyond  the  mathematical  details  of  the 
training  problems,  Aey  found  poor  performance  on  far 
transfer  problems.  The  elaborations,  though,  may  have 
failed  to  provide  support  for  subgoal  learning.  The  results  of 
the  present  study  suggest  that  a  large  amount  of  elaboration 
is  certainly  not  the  key  to  improving  transfer  from  examples 
(see  also  Kieras  &  Bovair,  1984);  rather,  the  additional 
information  can  be  quite  minimal  if  it  focuses  on  the  right 
kind  of  knowledge.  This  knowledge  can  be  fruitfully  con¬ 
ceptualized  as  subgoals. 


Implications 

The  results  strongly  suggest  that  learners  are  often  unable 
on  their  own  to  make  a  set  of  mathematical  steps  meaning¬ 
ful.  Rather,  students  need  help  from  either  a  teacher  or 
textbook.  This  is  an  important  finding  because  some  edu¬ 
cators  may  have  a  tendency  to  assume  that  the  meaning  of 
a  set  of  steps  is  envious  and  that  the  students  will  surely 
recognize  their  overall  purpose.  This  assumption  may  be 
wrong  much  more  often  than  educators  would  like  to 
believe. 

Even  when  educators  recognize  the  value  of  subgoals, 
they  might  not  be  skilled  at  identifying  the  best  ones  for  a 
particular  problem-solving  domain,  especially  in  areas  such 
as  math  and  physics.  For  instance,  Chi  et  al.  (1989,  p.  149) 
discussed  a  problematic  mechanics  example  from  a  text¬ 
book.  In  the  example,  a  block  was  suspended  from  a  ceiling 


by  two  pieces  of  rope  joined  at  a  knot  and  by  a  third  piece 
of  rope  going  from  the  knot  to  the  block.  The  task  was  to 
find  the  magnitude  of  two  of  the  forces  given  the  third  force. 
The  solution  states  that  the  knot  where  the  three  strings  are 
joined  should  be  considered  the  body.  However,  no  expla¬ 
nation  was  given  as  to  why  this  decision  was  made.  The 
decision  was  made  because  to  find  a  force  in  terms  of  other 
forces,  the  student  must  determine  that  the  forces  act  on  a 
common  point.  In  this  problem  the  only  place  where  all 
three  forces  act  was  the  Imot.  This  critical  subgoal  of  finding 
a  common  point  where  the  forces  were  acting  was  informa¬ 
tion  that  would  have  been  useful  for  students  to  have  when 
solving  funire  problems.  However,  instead  of  conveying 
this  subgoal,  the  example  was  more  likely  to  convey  a  series 
of  steps  that  may  or  may  not  have  been  useful  for  other 
problems. 

There  may  not  exist  a  correct  set  of  subgoals  to  be  learned 
for  a  particular  domain.  Researchers  might  show  one  set  to 
be  more  effective  than  another  by  looking  at  the  problem¬ 
solving  performance  of  students  taught  one  set  or  the  other. 
Thus,  educators  who  differ  in  opinion  about  the  usefulness 
of  certain  subgoals  can  explicitly  compare  their  subgoal  sets 
because  the  educators  are  at  least  using  a  common  cognitive 
language. 

The  present  study  provides  some  information  on  this 
issue.  Across  the  studies  most  students  formed  the  subgoal 
to  find  A.  However,  this  subgoal  was  not  all  that  uMful  to 
the  students  for  solving  novel  transfer  problems.  If  it  were, 
Aen  students  in  the  no-label  group  should  have  successfully 
found  A  in  those  problems  as  often  as  did  the  other  smdents. 
That  the  no-label  group  did  not  suggests  that  learning  the 
lower  level  subgoal  of  finding  total  frequency  was  crucial 
for  transfer  success.  It  is  not  clear  whether  an  appropriate 
level  can  be  proven  or  derived  with  a  logical  analysis  or  a 
cognitive  architecture,  but  it  is  an  interesting  issue  for  future 
research. 

Besides  differing  on  which  subgoals  they  believe  are  the 
most  useful  to  teach,  educators  and  researchers  might  also 
be  unclear  about  the  best  ways  to  aid  subgoal  learning.  The 
current  results  indicate  that  labeling  and  visual  isolation  can 
be  effective  techniques. 


Extensions 

Whereas  the  current  results  suggest  that  examples  can  be 
improved  to  aid  problem-solving  transfer,  Chi  and  her  col¬ 
leagues  (e.g.,  Chi  &  Bassok,  1989;  Chi  &  VanLehn,  1991) 
suggested  that  educators  can  improve  learning  by  teaching 
students  to  produce  better  self-explanations.  Chi  showed 
that  individuals  vary  in  terms  of  what  kinds  of  information 
they  extract  from  examples.  In  fact,  students  in  the  present 
study  may  have  varied  in  what  they  learned  from  examples 
regardless  of  the  experimental  manipulations.  That  is,  stu¬ 
dents  who  leam  easily  could  have  learned  the  goal  to  find 
the  total  frequency  on  their  own  when  studying  the  exam¬ 
ples  regardless  of  the  manipulation,  whereas  the  students 
who  have  difficulty  learning  might  have  been  less  likely  to 
do  so  (e.g.,  Chi  et  al.,  1989).  Unfortunately,  no  a  priori 
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infoTmation  was  collected  by  which  to  classify  students  in 
the  experiments  presented  here.  As  a  result,  it  is  possible 
that  the  effects  of  the  manipulations  might  have  been 
clouded.  In  future  experiments  similar  to  the  ones  presented 
here,  researchers  could  segregate  students  into  those  who 
learn  easily  and  those  who  do  not  (by  means  of  SAT  scores 
or  performance  on  some  prior  task)  and  examine  whether 
the  manipulations  affect  both  groups  in  the  same  way. 

Researchers  could  further  test  the  subgoal  approach  by 
teaching  learners  subgoals  that  are  hypothesized  not  to 
match  well  with  the  subgoals  needed  to  solve  test  problems. 
These  learners  should  perform  as  poorly  as,  or  perhaps 
worse  than,  learners  who  simply  memorized  a  set  of  steps. 
That  is,  if  subgoals  are  used  to  guide  problem  solving 
performance,  then  inappropriate  subgoals  should  hinder 
transfer. 

It  is  suggested  here  that  an  important  factor  for  educators 
interested  in  promoting  transfer  is  to  ensure  that  somehow 
the  relevant  subgoals  for  a  domain  are  conveyed  to  learners. 
This  implies  that  a  crucial  early  step  in  teaching  problem 
solving  in  a  domain  is  for  teachers  to  spend  time  identifying 
to  themselves  the  useful  subgoals  from  a  novice’s  perspec¬ 
tive.  How  might  this  be  done?  One  possibility  is  to  first 
identify  a  target  set  of  problems  that  the  instructor  wants  the 
students  to  be  able  to  solve.  Then  the  instructor  should  write 
out  the  solutions  to  these  problems  and  analyze  them  to 
determine  the  subgoals  achieved  by  groups  of  steps  that 
constitute  the  solution  procedures  to  the  problems.  Clearly, 
it  will  be  important  for  researchers  to  find  a  standardized 
way  of  identifying  subgoals  that  are  to  be  taught.  This  might 
be  a  difficult  task,  but  one  that  could  greatly  benefit  teach¬ 
ing  and  learning. 
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Appendix  A 

Training  Example  Used  in  Experiment  1  With  No-Label  and  Label  Solutions 
A  i«lp  to.  of  to  219  »  City  Ihl]  9^?*. 

to  "f* 

briefcases. 

a.  No-Label  Solution: 

1(180)  +  2(17)  +  3(13)  +  4(9)  289 


E(X)  =  - 


P(X  =  Jt)  = 


219 


289 

_  __  =  1  32  =  A  =  average  number  of  briefcases  owned  per  lawyer 
219 


[(e"*KA‘)l 


*! 


[(2.718-'-'^)(1.322)]  (.27)(1.74)_ 

P(X  =  2)  = - - - - - 2 - - 

Jf' 

b.  Label  Solution: 

Total  number  of  briefcases  owned  =  [1(180)  +  2(17)  +  3(13)  +  4(9)]  —  289 
289 

g/jQ  _ _ =  1  32  =  A  =  average  number  of  briefcases  owned  per  lawyer 

219 


P(X  =  jc)  = 


[(e"*KA')] 


x! 


[(2.718"'-^^)(1.32^)]  (.27)(1.74)  _ 

P(X  =  2)  = - - - - - 2 


Appendix  B 

Test  Problem  in  Which  Total  Frequency  Is  Given  Directly 

A  number  of  celebrities  were  asked  how  many  commercials  they  i^e  over  the  last  year,  ^e  20 
celebrities  ma/t..  a  total  of  71  commercials.  Use  the  Poisson  ^stribution  to  determme  the  probabihty 
diat  a  randomly  chosen  celebrity  made  exactly  5  commercials. 

Solution  (not  seen  by  students): 

71 

n/y\  — _ =:3  55  =  A  =  average  number  of  commercials  per  celebrity 

^  ^  20 


P(X  =  5)  = 


[(2.718~^-^^)(3.55^)]  _  (.029)(563.8) 


5! 


120 


=  .135 


Appendix  C 

Test  Problem  in  Which  Total  Frequency  Is  Calculated  by  Adding  Simple  Frequencies  (Expenment  2) 

Over  the  course  of  the  summer,  a  group  of  5  kids  used  to  walk  along  the  ^ch  Mch  day 
seasheUs  We  know  that  on  Day  1  Joe  found  4  shells,  on  Day  2  Sue  found  2  shells,  on  Day  3  Mary 
Sd  5  sheik  orDT4  Roger  found  3  shells,  and  on  Day  5  Bill  found  6  sheUs  Use  the  Poisson 
distrfbution  to’deteiJne  the  probability  of  a  randomly  chosen  kid  finding  3  shells  on  a  particular 

day. 
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Test  Problem 


Instance 


Solution  (not  seen  by  students): 

4  +  2  +  5+3  +  6  20 

E(X)  = - - - 


=  —  =  4.0  =  A  =  average  number  of  shells  per  kid 


r(2.718-">)(4.(P))  <.018X64) 
IXX-S) - 51 - ; - 


.195 


Appendix  D 

With  Weighted  Average  Method  to  Find  X  but  in  Which  Trials  (Denominator)  Must  Be  Found 
by  Adding  Number  of  Members  in  Each  Category  (Experiment  2) 

A  construction  crew  had  a  varying  number  of  people  who  knew  how  to  use  a  jackhaiMer, 

^twnHino  on  the  particular  job  that  was  needed.  On  10  of  the  jobs  they  did,  only  one  person  toew 
how  to  use  a  jackhammer,  on  13  of  the  jobs  2  people  knew  how  to  use  jackhammers,  on  6  of  the 
jobs  3  people  knew  how  to  use  jackhammers,  and  on  7  of  the  jobs  4  people  knew  how  to  use 
jackhammers.  Use  the  Poisson  distribution  to  determine  the  probability  of  exactly  two  people  m  the 
crew  knowing  how  to  use  a  jackhammer  on  a  randomly  chosen  job. 


Solution  (not  seen  by  students): 


E(X)  = 


1(10)  +  2(13)  +  3(6)  +  4(7) 
10+13  +  6  +  7 


—  =  2  28  =  A  =  average  number  of  “knowers”  per  job 
36 


[(2.718”^“X2.28^)] 

P(X  =  2)  = - ;; - 


(■102)(5.2) 

2 


.265 


Appendix  E 

of  Example  Solution  Studied  by  the  Visual  Isolation  and  Same  Line  Groups  (Experiment  3) 

a.  Visual-Isolation  Group 
1(180)  +  2(17)  +  3(13)  +  4(9)  =  289 
289 

R/jA  =  —  =  1  32  =  A  =  average  number  of  briefcases  owned  per  lawyer 
^  219 

[(e"*X^*)] 

P(X  =  a:)  =  — — 


P(X  =  2)  = 


[(2.718-*^^)(1.32^)]  (■27)(1.74)  _ 


2! 


.235 


b,  Same-Line  Label  Group 

1(180)  +  2(17)  +  3(13)  +  4(9)  total  number  of  briefcases  owned  ^  289 
E(X)  —  219  219 

=  1.32  =  A  =  average  number  of  briefcases  owned  per  law7er 


P(X-x)  =  - 


[(^■*XA0] 


x\ 


P(X-2)  =  - 


[(2.718‘'-^2)(i  322)]  (.27)(1.74) 


2! 


=  .235 
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Abstract 

Three  exceriments  tested  the  hypothesis  that  when  learners  are  led  to  group  steps  from 
example  rolutions,  they  will  be  more  likely  to  learn  subgoals  that  can  be  ttansferred  to 
novelVwems,  thereby  improviilg  problem  solving  The  reidu  from  e^n^nt 
suggest  that  a  label  serves  as  a  cue  for  grouping  by  demonstrating  that  a  reiatweiy 
mlmiingless  label  in  an  example  solution  was  as  effective  (and 
effective)  as  a  meaningful  label  in  helping  learners  transfer  to  novel  proWems. 
Experiment  2  provided  converging  evidence  for  subgoal  learmng 
oa^cioants  stodying  example  solutions  with  a  label  were  more  likely  to  segment  Ae 
KHs  a  Son  of  J  label  and  were  more  likely  to  mention  the  eorrespondmg 
subgoal  in  their  descriptions  of  how  to  solve  problems. 


Introduction 

A  good  deal  of  research  has  examined  the  transfer  success  people  have  after  studying 
training  materials  such  as  those  containing  step-by-step  instructions  (Kieras  &  Bovair.  1984;  Smith 
&  Goodman,  1984),  examples  (e.g.,  Ross,  1987, 1989),  or  both  (Fong,  Krantz,  &  Nisbett, 

1986).  Although  there  have  been  some  exceptions  (e.g.,  Fong  et  al.,  1986;  Zhu  &  Simon,  1987), 
the  usual  finding  from  such  research  is  that  people  can  carry  out  new  procedures  or  solve  new 
problems  that  are  quite  similar  to  those  on  which  they  were  trained,  but  have  difficulty  when  the 
novel  cases  involve  more  than  minor  changes  from  what  they  had  previously  studied. 

This  transfer  difficulty  seems  to  stem  from  a  tendency  by  many  learners  to  form 
representations  of  a  solution  procedure  that  consist  of  a  Unear  series  of  steps  rather  than  a  more 


^Currently  under  review. 
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t  structured  hierarchy.  An  advantage  of  a  hierarchical  organization  is  that  it  can  provide  guidance  for 
fiHapfing  the  proceduTB  for  novel  cases.  One  potentially  useful  hierarchical  organization  for  a 
solution  procedure  would  be  a  set  of  goals  and  subgoals  with  methods  for  achieving  them  (e.g., 
Anzai  &  Simon,  1979;  Card,  Moran,  &  Newell,  1983;  Catrambone  &  Holyoak,  1990;  NeweU  & 
Simon,  1972;  Singley  &  Anderson,  1989).  Problems  within  a  domain  typically  share  the  same  set 
of  subgoals,  although  the  steps  for  achieving  the  subgoals  might  vary  from  problem  to  problem. 
For  instance,  physics  mechanics  problems  typically  share  the  subgoals  of  identifying  all  "systems" 
in  the  problem  and  identifying  aU  forces  acting  on  the  object  of  interest  regardless  of  whether  the 
^problems  involve  objects  on  inclined  planes  or  blocks  suspended  over  pulleys  (Heller  &  Reif, 
^984). 

Consider  a  student  facing  a  novel  problem,  that  is,  one  in  which  the  steps  are  not  the  same 
as  those  seen  in  a  previously-studied  example.  If  the  student  has  memorized  only  a  rote  set  of 
steps  for  the  overall  solution  procedure,  he  or  she  will  have  little  guidance  as  to  which  steps  need 
to  be  modified,  as  well  as  what  new  steps  might  need  to  be  created,  in  order  to  solve  the  problem. 
Conversely,  a  student  who  learned  a  solution  procedure  organized  by  subgoals  and  methods  could 
attempt  to  apply  those  subgoals  to  the  novel  problem.  This  approach  has  two  advantages.  First, 
the  learner  would  know  which  steps  from  the  learned  procedure  are  relevant  for  achievmg  a 
particular  subgoal.  Thus,  if  those  exact  steps  can  not  be  carried  out  in  the  current  problem,  the 
learner  knows  which  set  of  steps  on  which  to  focus  for  modification.  Second,  if  the  learner  is 
attempting  to  achieve  a  particular  subgoal  and  realizes  that  a  modification  to  the  old  steps  will  not 
^achieve  the  subgoal; then  the  subgoal  can  help  constrain  the  memory  search  for  other  relevant 
-information  for  achieving  that  subgoal  (Anzai  &  Simon,  1979).  Thus,  the  search  space  for  useful 
information  would  be  reduced  for  the  second  learner. 

Forming  Hierarchical  Organizations  for  Solving  Problems 

A  variety  of  studies  have  examined  the  effects  on  problem  solving  for  learners  who  form  a 
hierarchical  structure  representing  a  solution  procedure  (e.g.,  Dufresne,  CJerace,  Hardiman,  & 
Mestre,  1992;  Eylon  &  Reif,  1984).  The  typical  result  was  that  learners  were  able  to  solve  novel 
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problems  more  successfully  than  learners  who  were  led  to  form  a  single-level  organization  of  the 
problem  solving  procedure.  In  these  studies,  researchers  usually  derived  what  they  beUeved  to  be 
a  useful  hierarchical  approach  to  problem  decomposition  and  induced  learners  to  mtemalize  this 
approach  by  following  a  prescribed  method  for  solving  or  processing  training  problems.  Although 
these  hierarchical  approaches  were  not  always  couched  in  terms  of  subgoals  and  methods,  they 
could  certainly  be  viewed  in  this  light. 

For  instance,  HeUer  and  Reif  (1984)  formulated  a  model  specifying  the  underlying 
knowledge  and  procedures  needed  to  successfuUy  solve  mechanics  problems.  This  procedural 
knowledge  included  drawing  force  diagrams  for  all  external  forces  in  a  problem  and  identifying 
"short-range"  and  "long-range"  forces.  The  authors  required  participants  to  solve  three  problems 
.  by  adhering  to  a  hierarchical  model  (the  "M"  model)  for  redescribing  each  problem  in  terms  of 
.relevant  forces.  This  model  was  designed  to  address  various  deficiencies  learners  demonstrate 
when  attempting  to  solve  mechanics  problems.  The  model  was  contrasted  with  the  M*  model  that 
intentionaUy  omitted  certain  levels  of  the  redescription  process  hierarchy  such  as  checking  for 
consistency  between  the  direction  of  forces  and  the  resulting  acceleration  (learners  were  not 
prevented  from  doing  these  checks,  but  they  were  not  reminded  to  do  them).  Participants  who 
,,were  required  to  foUow  the  M  model  performed  significantly  better  at  redescribing  novel  problems 

and  solving  them. 

Heller  and  Reif  (1984)  were  quick  to  point  out  that  the  M  model  was  only  prescriptive  and 
did  not  necessarily  have  any  direct  relationship  to  internal  representations  learners  might  have 
formed  by  following  the  model.  It  is  also  the  case  that  besides  containing  certain  high-level 
subgoals  such  as  describing  motion  that  the  M*  model  lacked,  the  M  model  included  a  number  of 
low-level  "hints,"  such  as  reminding  learners  to  include  properties  such  as  mass  in  their  drawings, 
that  the  M*  model  did  not  include.  Thus,  it  is  not  entirely  clear  whether  requiring  participants  to 
focus  on  a  certain  prescribed  set  of  subgoals  was  most  responsible  for  superior  performance  on 
new  problems  or  whether  the  lower-level  procedural  details  were  also  crucial. 
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Brown,  Kane,  and  Echols  (1986)  found  a  similar  result  in  the  area  of  short  story  problems. 
fhildfRn  demonstrated  better  spontaneous  transfer  to  novel  problems  if  they  had  focused  (either  on 
their  own  or  through  an  experimental  intervention)  on  a  hierarchical  structure,  in  this  case  the  goal 
structure,  of  the  base  analog. 

Subgoal  Learning 

TTie  studies  mentioned  above  focusing  on  hierarchy  learning  used  fairly  powerful 
manipulations  to  induce  learners  to  form  different  types  of  organizations.  While  this  approach  can 
»yield  practical  implications  for  instructional  manipulations,  it  provides  fewer  constraints  for  models 
%f  problem  solving  since  it  can  be  difficult  to  determine  which  features  of  the  manipulation  led  to 
-performance  differences. 

The  purpose  of  the  present  research  was  to  test  a  model  of  subgoal  learning  by  examining 
the  transfer  effects  due  to  a  manipulation  designed  to  influence  whether  learners  formed  a  particular 
subgoal. 

In  the  present  work  the  assumption  is  made  that  learners’  knowledge  for  solving  problems 
in  a  domain  can  be  fruitfully  represented  in  terms  of  subgoals  with  steps  for  achieving  them.  At 
one  extreme  a  learner  could  have  a  single  goal;  to  solve  a  particular  problem  type.  This  goal 
would  be  achieved  by  a  linear  series  of  steps.  Towards  the  other  extreme  would  be  a  solution 
procedure  broken  into  a  number  of  subgoals,  each  with  an  associated  method  for  achieving  it. 

Each  method  would  consist  of  a  small  number  of  steps  relative  to  the  total  number  of  steps  in  the 
entire  solution  procedure. 

^  Transfer  to  novel  problems  is  assumed  to  be  at  least  partly  a  function  of  applying  subgoals 

'learned  from  prior  problems  or  examples  to  a  new  problem.  Thus,  manipulations  designed  to 
affect  whether  or  not  a  particular  subgoal  is  learned  should  affect  a  person  s  ability  to  solve  novel 
problems. 

The  notion  of  a  subgoal  is  sometimes  applied  to  a  construct  generated  by  a  learner  when  he 
or  she  reaches  in  impasse  during  problem  solving  (e.g.,  Newell,  1990,  Chap  4;  VanLehn,  1988) 
and  is  sometimes  used  to  refer  to  a  feature  of  a  task  structure  that  can  be  taught  to  a  learner  (e.g., 
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Catrambone  &  Holyoak.  1990;  Dixon,  1987).  Both  views  converge  on  the  prediction  that  learning 
subgoals  can  help  one  transfer  more  successfully  to  novel  problems.  The  factors  that  affect 
subgoal  learniug  remain  unclear  though. 

Factors  Influencing  Subcoal  Learning 

Many  models  of  transfer  posit  the  existence  of  subgoals  but  do  not  necessarily  explain  how 
they  are  learned  or  formed.  Anzai  and  Simon  (1979)  offered  an  account  of  subgoal  learning  in  the 
.  context  of  a  person  learning  to  solve  the  Tower  of  Hanoi  problem.  They  recorded  the  moves  and 
verbal  protocol  of  the  learner  as  she  solved  the  problem  multiple  times.  Two  observations  from 
that  study  are  related  and  particularly  relevant  to  the  present  research.  The  first  is  that  over  trials 
the  learner  began  to  chunk  groups  of  moves,  that  is,  she  would  make  a  set  of  moves  m  quick 
.succession  foUowed  by  a  pause  before  the  next  set  of  moves.  The  second  is  that  she  appeared  to 
,  form  goals  and  subgoals  in  her  representation  of  the  procedure  for  solving  the  Tower  of  Hanoi 

problem. 

Anzai  and  Simon  argued  that  subgoal  acquisition  is  not  trivial  and  its  occurrence  is  greatly 
aided  when  the  search  space  (e.g.,  possible  moves  in  the  Tower  of  Hanoi  problem)  is  simplified. 
This  simpUfication  will  frequently  initially  require  prior  knowledge  of  the  learner  of  certain  facts 
that  can  be  applied  to  the  domain.  In  the  case  of  the  Tower  of  Hanoi,  such  a  fact  might  be  that 
move  repetitions  are  inefficient.  When  the  search  space  is  simpUfied,  working  memory  load  is 

reduced. 

One  hypothesized  advantage  of  a  working  memory  load  reduction  is  that  the  learner  is 
better  able  to  notice  and  remember  a  sequence  of  steps  that  led  to  a  particular  outcome  (see  also 
Sweller,  1988).  In  Anzai  and  Simon's  model  this  aids  subgoal  formation  because  a  subgoal  is 
formed  when  a  learner  is  working  towards  a  certain  goal  (perhaps  derived  from  task  instructions) 
and  notices  that  a  set  of  steps  places  him  or  her  in  a  situation  to  be  able  to  carry  out  additional  steps 
that  ultimately  achieve  the  goal.  The  learner  will  be  better  able  to  notice  the  result  of  the  first  set  of 
steps,  and  be  able  to  chunk  that  sequence  of  steps,  if  working  memory  load  has  been  reduced. 
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A  key  component  in  Anzai  and  Simon’s  model  is  the  presence  of  a  perceptual  system  that 
allows  the  learner  to  observe  various  external  features  of  the  problem  situation.  In  the  case  of  the 
Tower  of  Hanoi  problem,  one  feature  would  be.  for  example,  a  particular  disk  being  located  next 
to  a  smaller  disk.  However,  in  learning  tasks  that  are  less  obviously  perceptually  oriented,  such  as 
learning  to  solve  word  problems  in  probability,  physics,  or  algebra,  simple  perceptual  features  are 
less  likely  to  play  a  key  role  in  subgoal  formation.  Rather,  cues  in  worked  examples  and  the 
learner’s  background  knowledge  will  play  a  larger  role.  Tbese  cues  may  take  the  form  of  text  and 
diagrams  in  the  problem  that  direct  the  learner  to  relevant  aspects  of  the  problem  and  relevant  pnor 
♦  knowledge  (cf.  Ward  &  Sweller,  1990).  The  cues  can  direct  the  learner  to  group  a  set  of  steps  m 
the  example  solution  and  thus,  to  increase  his  or  her  chances  of  recognizing  that  a  particular 
outcome  is  the  result  of  the  execution  of  those  steps.  That  is.  the  recognition  of  the  grouping  is 
hypothesized  to  lead  the  learner  to  try  to  uncover  the  purpose  of  the  group  of  steps.  This 
"purpose”  can  be  conceptualized  as  a  subgoal.  One  such  cue  that  could  encourage  grouping,  and 

thus  subgoal  learning,  is  a  label. 

tiHr.wi.nri7aion.  Wattenmaker,  Dewey,  Murphy,  and  Medin  (1986)  found  that 
providing  learners  with  a  theme  (e.g.,  think  of  objects  in  one  category  as  being  or  not  being 
reasonable  substitutes  for  a  hammer)  during  a  training  session  helped  them  learn  categories  more 
quickly  (see  also  Cabrera  &  BiUman,  1994;  Homa  &  Cultice,  1984).  Medin,  Wattenmaker,  and 
Hampson  (1987)  found  that  learners  tended  to  sort  a  set  of  items  around  a  smgle  pnmary 
dimension  or  sometimes  correlated  features  for  which  causal  or  explanatory  links  could  be  readily 
;  made.  When  themes  (e.g.,  flying)  were  made  more  salient  to  learners,  they  tended  to  categorize 
more  on  famUy  resemblance  than  individual  features.  These  results  suggest  that  features  of 
example  solutions  that  help  learners  form  links,  such  as  causal  or  explanatory  ones,  among  items 

will  help  learners  form  a  category  that  captures  a  useful  relationship  about  the  items.  Thus,  with 
respect  to  learning  solution  procedures,  cues  that  help  learners  determine  which  steps  go  together 
and  what  their  purpose  is,  might  aid  the  formation  of  a  subgoal  representing  the  steps'  purpose. 


Page  32 


The  hypothesized  benefits  of  labels  for  subgoal  learning  are  also  consistent  with  Fried  & 

Holyoak's  (1984)  category  density  model.  Their  model  assumes  that  the  goal  of  the  category 
learner  is  to  develop  a  schematic  description  of  the  distributions  of  category  exemplars  over  a 
feature  space.  Highly  salient  features  will  tend  to  be  encoded  initially  since,  in  an  unfamihar 
domain,  the  learner  will  probably  have  trouble  identifying  less  salient  features.  In  problem 
solving,  these  less-salient  features  constitute  the  deep  structure  of  problems.  For  instance,  when 
learners  study  examples  in  domains  such  as  prob^ility  and  physics,  they  tend  to  focus  on  familiar 
surface  features  such  as  basic  mathematical  operations  and  objects  (e.g.,  multiplying  a  set  of 
numbers,  blocks  on  inclined  planes)  rather  than  goals  being  achieved  by  the  operations  such  as 
determining  the  forces  acting  upon  the  object  of  interest  (Chi,  Feltovich,  &  Glaser,  1981). 
-Manipulations  that  help  learners  focus  on  meaningful  sets  of  steps  could  lead  learners  to  discover 

'•  the  emergent  deep  structure. 

How  Subgoals  Aid  Transfer  to  Novel  Problems 

Wattenmaker  et  al.  (1986)  suggested  that  the  degree  of  difficulty  in  learning  a  particular 
category  structure  is  at  least  partly  a  function  of  the  type  of  knowledge  that  learners  bring  to  the 
task.  With  respect  to  the  present  work,  subgoals  influence  the  knowledge  a  learner  brings  to  the 
task  of  solving  a  novel  problem  by  reducing  the  search  space  that  is  explored  while  the  learner  tries 
to  modify  an  old  solution  procedure. 

A  particular  subgoal  will  have  a  set  of  steps,  or  method,  associated  with  it,  and  if  the 
learner  discovers  that  those  steps  can  not  be  used  on  a  particular  problem,  he  or  she  will  have  a 
reduced  search  space  to  consider  when  trying  to  adapt  the  method.  That  is,  the  learner  knows  on 
which  steps  to  focus  for  changing  the  procedure.  Conversely,  a  learner  who  has  learned  a  solution 
procedure  consisting  of  a  single  goal  with  a  long  series  of  steps  will  be  less  likely  to  determme 
successfiiUy  which  steps  need  to  be  modified  in  order  to  solve  the  problem  because  the  search 
space  of  possible  modifications  will  be  larger.  In  a  more  extreme  case,  if  a  particular  subgoal 
needs  to  be  achieved  in  a  very  different  way  than  was  demonstrated  in  the  example  (i.e.,  new  steps 
are  required  rather  than  a  modification  of  old  steps),  the  learner  possessing  a  representation  with 
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subgoals  will  have  some  guidance  about  what  prior  knowledge  might  be  relevant  to  achieving  the 
subgoal.  The  learner  who  memorized  only  a  series  of  steps  will  be  less  likely  to  know  what 
knowledge  he  or  she  possesses  might  be  useful.  Thus,  a  person  representing  a  solution  procedure 
in  terms  of  subgoals  and  steps  for  achieving  the  subgoals  has  a  more  flexible  procedure  than  a 
person  representing  the  procedure  as  a  single  long  series  of  steps. 

For  instance,  in  the  probability  exan:q)les  used  in  the  current  study,  the  ultimate  goal  of  each 
problem  is  to  calculate  a  probability.  The  solution  procedure  for  achieving  this  goal  involves  a 
number  of  steps,  a  subset  of  which  constitutes  a  sequence  of  multiplication  and  addition  operations 
that  can  be  grouped  under  the  subgoal  "find  the  total  frequency  of  the  event. 

Consider  the  "No  Label"  solution  to  the  probability  example  in  Table  1  involving  the 
Poisson  distribution.  ^  A  learner  could  study  this  example  and  memorize  the  steps  for  solving  a 
problem  that  involves  the  same  set  of  steps  even  if  the  new  problem  involved  farmers  and  tractors 
instead  of  lawyers  and  briefcases.  After  studying  the  No  Label  solution,  the  learner's  knowledge 
for  the  part  of  the  solution  procedure  that  involves  finding  X,  the  average,  might  be  represented  as: 

Goal:  FindX 

Method:  1.  Multiply  each  category  (e.g.,  owning  exactly  zero  briefcases, 
owning  exactly  one  briefcase,  etc)  by  its  observed  frequency. 

2.  Sum  the  results. 

3.  Divide  the  sum  by  the  total  number  of  lawyers  to  obtain  the 
average  number  of  briefcases  per  lawyer. 

This  representation  would  serve  the  learner  well  for  problems  that  involve  calculating  X  in 
the  same  way  as  the  example.  However,  this  representation  fails  to  capture  the  fact  that  the  first 
Une  of  the  No  Label  solution  in  Table  la  also  involves  calculating  a  total  frequency.  Finding  the 
total  frequency  is  a  subgoal  that  might  be  achieved  in  a  variety  of  ways  depending  on  the  givens  in 
the  problem.  A  novel  problem  that  requires  finding  total  frequency  in  a  different  way  than  in  the 
example  might  cause  problems  for  the  learner  with  the  above  representation.  For  instance, 
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consider  the  problem  in  Table  2a.  In  this  problem  the  total  frequency  is  calculated  by  adding  a  set 
of  simple  frequencies.  This  is  a  less-complex  method  than  was  used  in  the  example,  but  the 
learner  might  not  be  able  to  construct  it  because  the  subgoal  for  finding  the  total  frequency,  and  an 
instance  of  a  method  for  achieving  it,  were  never  isolated.  If  the  learner  had  formed  the  foUowmg 
representation,  then  his  or  her  chance  of  solving  the  problem  in  Table  2a  might  be  better  smee  this 
representation  identifies  the  steps  involved  in  finding  the  total: 

Goal:  Find^ 

Method:  1  -  Goal:  Find  pnmher  of  briefcases 

Method:  a.  Multiply  each  category  by  its  observed 
frequency. 

i  b.  Sum  the  results  to  obtain  the  total  number 

of  briefcases. 

2.  Divide  the  total  number  of  briefcases  by  the  total  number  of 
lawyers  to  obtain  the  average  number  of  briefcases  per  lawyer. 


Insert  Tables  1  and  2  about  here 


Catrambone  (in  press)  found  that  learners  studying  the  "Meaningful  Label"  solution  (Table 
lb)  to  the  exaiiq)le  were  more  likely  than  No  Label  learners  (who  studied  the  solution  in  Table  la) 
to  find  the  total  frequency  as  measured  by  their  success  at  solving  problems  such  as  the  one  m 
Table  2a.  This  was  taken  as  initial  evidence  that  the  former  group  had  learned  the  subgoal  to  find 

the  total  frequency. 

The  account  proposed  in  Catrambone  (in  press)  for  why  the  Label  group  was  mote  likely 
than  the  No  Label  group  to  leam  the  subgoal  to  find  the  total  ftequency  could  be  summarized  as 

follows: 
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^  1)  A  label  leads  learners  to  group  a  set  of  steps  (such  as  the  steps  for  finding  the 

total  frequency). 

2)  After  grouping  the  steps,  learners  are  likely  to  try  to  self-explain  why  those  steps 
go  together. 

3)  The  result  of  the  self-explanation  process  is  the  formation  of  the  goal  that 
represents  the  purpose  of  that  set  of  steps. 

While  most  learners  can  presumably  engage  in  a  self-explanation  process,  with  varying 
degrees  of  success,  good  students  seem  better  at  determining  the  appropriate  boundaries  between 
*  meaningful  groups  of  steps  in  a  solution  procedure  (Chi,  Bassok,  Lewis,  Reimann,  &  Glaser, 
1989).  The  use  of  a  label  in  examples  is  hypothesized  to  serve  as  a  cue  to  the  boundaries.  Thus, 

'  Label  participants  in  Catrambone  (in  press)  were  helped  presumably  to  focus  on  the  steps  that 
formed  a  coherent  unit  (finding  a  total)  whereas  the  No  Label  participants  were  not. 

While  the  results  of  Catrambone  (in  press)  were  consistent  with  the  above  account,  they  did 
not  constitute  a  strong  test  of  the  account.  That  is,  it  is  possible  that  part  of  the  transfer  advantage 
enjoyed  by  the  Label  group  could  have  been  due  to  the  fact  that  the  label  itself  provided  information 
beyond  serving  as  a  cue  to  group  a  set  of  steps.  That  is,  the  label  indicated  that  the  total  number  of 
briefcases  was  being  found.  Thus,  instead  of  the  label  leading  learners  to  group  a  set  of  steps  and 
inducing  a  self-explanation  process,  it  may  simply  have  provided  them  with  this  fact:  finding  the 
total  number  of  things  is  something  that  one  does  when  solving  Poisson  problems  (it  is  assumed 
that  college-aged  learners  are  sophisticated  enough  to  generahze  "bnefcase  to  things  ). 

One  way  to  tease  apart  these  possible  explanations  is  to  provide  learners  with  labels  that 
contain  no  expUcit  information  about  the  domain  and  examine  whether  transfer  performance  is  as 
good  as  transfer  performance  by  learners  who  study  examples  with  more  meaningful  labels.  This 

is  the  general  approach  taken  in  the  experiments  in  the  present  study. 

Experiment  1  explored  the  possibility  that  a  label  has  to  have  some  meaning  in  order  for  the 
learner  to  learn  a  subgoal  as  opposed  to  a  simply  serving  as  a  cue  for  the  learner  who  then  uses 
background  knowledge  to  construct  the  subgoal.  Experiment  2  sought  converging  evidence  for 
subgoal  learning  to  supplement  the  evidence  from  transfer  performance.  Experiment  3  explored 
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the  generality  of  the  subgoal  that  is  formed  from  a  semantically  meaningful  label  compared  to  one 
formed  from  a  less-meaningfiil  label. 

T  jjaming  from  Examples 

The  present  study  examined  learners  acquiring  subgoals  by  studying  examples  rather  than 
through  direct  instruction  of  those  subgoals.  There  were  two  primary  reasons  for  this  approach. 
First,  people  typically  prefer  to  study  and  use  worked  examples,  or  problems  they  have  previously 
solved,  when  attempting  to  solve  new  problems  (e.g.,  LeFevre  &  Dixon,  1986;  PiroUi  & 

Anderson.  1985)  despite  the  fact  that  they  are  frequently  misled  by  surface  features  of  the  examples 

and  problems  (e.g.,  Ross,  1987, 1989). 

Second,  there  is  evidence  that  students,  at  least  good  students,  can  derive  knowledge  from 
-  examples  that  they  did  not  or  could  not  acquire  from  explanatory  text,  even  if  the  text  was  formally 
^Tcomplete.  Chi  et  al.  (1989)  found  that  after  studying  a  text  on  mechanics,  good  and  poor  students 
(as  defined  by  a  subsequent  problem  solving  test)  seemed  to  possess  similar  declarative 
knowledge.  However,  after  studying  worked  examples,  good  students  were  more  likely  to  acquire 
knowledge  about,  among  other  things,  the  conditions  of  appUcation  of  actions  or  operators,  the 
consequences  of  these  actions,  and  the  relationship  of  the  actions  to  goals.  Chi  et  al.  suggested 
that  this  additional  problem  solving  or  procedural  knowledge  was  the  result  of  a  self-explanation 
process.  Good  students  were  far  more  likely  than  poor  students  to  produce  self-explanations 
leading  to  the  acquisition  of  this  knowledge.  One  reason  good  students  were  more  likely  to  do  this 
is  because  they  could  better  recognize  the  locations  in  examples  that  contained  unexpUcated  actions. 

From  an  educational  point  of  view,  there  are  two  responses  to  the  findings  of  Chi  et  al. 

One  is  to  find  ways  to  help  poor  students  improve  their  self-explanation  skills  so  that  they  can 
derive  more  from  examples.  The  second,  and  the  one  chosen  here,  is  to  find  ways  to  improve 
examples  to  help  all  learners  derive  useful  information,  such  as  subgoals,  from  examples.  In  the 
present  work,  this  is  done  through  cues  in  example  solutions  that  could  help  learners  group  steps 
that  they  otherwise  might  not  realize  were  related. 
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*  '  '  ’  -  Ejq)eriment  1 

Experiment  1  examined  the  role  of  label  meaningfulness  in  subgoal  learning.  In  this 
experiment  the  No  Label  group  studied  examples  demonstrating  the  weighted  average  method  for 
finding  X  (all  solutions  in  Table  1  demonstrate  this  type  of  solution).  The  Meaningful  Label  and 
Less-Meaningful  Label  groups'  examples  differed  in  that  the  steps  for  finding  the  total  frequency 
were  explicitly  labeled  rather  than  merged  with  the  ovaall  set  of  steps  for  finding  X  (see  Tables  lb 
and  Ic  for  instances  of  the  "Meaningful  Label"  and  "Less-Meaningful  Label"  solutions, 

-  respectively). 

Subgoal  learning  was  assessed  by  how  successfully  participants  found  X  on  test  problems 
such  as  those  in  Table  2.  The  presence  of  a  label  was  not  hypothesized  to  improve  learners  ability 
to  memorize  the  solution  procedure  since  they  were  allowed  to  work  at  their  own  pace.  Thus,  the 
No  Label  group  was  predicted  to  be  as  successful  as  the  label  groups  at  solving  transfer  problems 
that  were  isomorphic  to  the  training  examples.  If  learners  receiving  examples  with  labels  solve 
novel  problems  more  successfully  than  the  No  Label  group,  this  would  be  consistent  with  the 
hypothesis  that  the  presence  of  a  label,  rather  than  its  meaningfiilness,  was  sufficient  to  induce 
subgoal  learning  If  the  Meaningful  Label  group  outperforms  the  L^ss-Meaningful  Label  group, 
this  would  suggest  that  the  meaningfulness  of  the  label  affected  the  likelihood  of  forming  a 
domain-relevant  subgoal  or  aided  transfer  in  some  other  way. 

Method 

Participants.  Participants  were  100  students  recruited  from  an  introductory  psychology 
class  at  the  Georgia  Institute  of  Technology  who  received  course  credit  for  their  participation. 

None  of  the  students  had  taken  a  probability  course  prior  to  participating  in  the  experiment. 

Materials  and  Procedure.  AJQ  participants  initially  studied  a  cover  sheet  that  briefly 
described  the  Poisson  distribution  and  how  it  could  be  used  as  a  replacement  for  more  cumbersome 
techniques  for  calculating  probabilities  involving  events  that  could  be  categorized  as  successes  and 
failures.  The  Poisson  equation  was  presented  along  with  a  simplified  notion  of  a  random  variable. 
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Participants  were  randomly  assigned  to  one  of  three  groups.  The  Meaningful  Label  group 
=  34)  studied  three  examples  demonstrating  the  weighted  average  method  for  finding  X  in 
which  the  steps  for  finding  the  total  frequency  were  given  a  label  that  was  assumed  to  have 
meaning  to  the  participants  and  made  mathematical  sense  given  the  steps  that  preceded  it  (see  the 
-Meaningful  Label  Solution"  in  Table  lb  for  an  example).  The  Less-Meaningful  Label  group  (H  = 
34)  studied  examples  in  which  the  steps  for  finding  the  total  frequency  were  labeled  with  Q  which 
was  assur^  to  have  little  meaning  for  the  participants  in  the  context  of  the  examples  (see  the 
"Less-Meaningful  Label  Solution"  in  Table  Ic).  The  No  Label  group  (H  =  32)  studied  examples  in 
which  the  steps  for  finding  the  total  frequency  were  not  labeled  (see  the  "No  Label  Solution"  m 

Table  la). 

After  studying  the  examples,  participants  solved  six  test  problems.  The  first  two  required 
^  the  use  of  the  weighted  average  method  for  finding  X  (isomorphic  to  the  example  in  Table  1). 
These  problems  were  given  first  so  that  participants  would  be  able  to  immediately  see  that  the  prior 
examples  were  relevant  for  solving  the  test  problems.  The  third  and  fourth  problems  provided  the 
total  frequency  directly,  thus  X  could  be  found  by  simply  dividing  the  given  total  frequency  by  the 
total  number  of  trials.  These  problems  were  the  problem  in  Table  2b  and  another  problem 
isomorphic  to  it.  The  fifth  and  sixth  problems  involved  adding  simple  frequencies  m  order  to  find 
the  total  frequency  (see  Table  2a  for  one  of  the  problems).  Participants  were  told  not  to  look  back 

at  the  examples  when  solving  the  test  problems. 

Participants’  written  solutions  were  scored  for  whether  they  found  X  correctly. 

Design.  The  independent  variable  was  type  of  example  solution  studied  (Meaningful 
Label,  Uss-Meaningful  Label,  No  Label),  thus  there  were  three  groups  in  the  experiment.  The 
dependent  measure  was  performance  on  the  six  test  problems,  two  of  which  were  isomorphic  to 
the  examples  while  the  other  four  were  novel. 

Results 

Participants  were  given  a  score  of  1  for  a  given  problem  if  they  found  X  correctly  and  a 
score  of  0  otherwise.  The  scores  for  the  two  problems  that  were  isomorphic  to  the  training 
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examples.  Problems  1-2,  were  summed,  thus  creating  a  score  from  0-2  for  performance  on  those 
problems.  Similarly,  the  scores  for  the  four  novel  problems.  Problems  3-6,  were  summed,  thus 
creating  a  score  from  0-4  for  performance  on  those  problems. 

As  expected,  all  groups  did  quite  well  at  finding  X  on  test  problems  that  were  isomorphs  to 
the  training  examples  (Problems  1-2).  All  participants  except  two,  one  in  the  Less-Meaningful 
Label  group  and  one  in  the  No  Label  group,  solved  both  isomorphs  correctly. 

There  were  significant  differences  among  the  three  groups  with  respect  to  finding  X.  in  the 
%novel  test  problems,  E(2, 97)  =  10.20,  .0001,  MSg  =  1.82,  with  means  of  3.44,  2.97,  and 

1.97  for  the  Meaningful  Label,  Less-Meaningful  Label,  and  No  Label  groups,  respectively.  The 
%most  typical  mistake  that  students  made  on  these  problems  was  to  write  in  the  solution  area  that  not 
enough  information  was  given  to  solve  the  problem.  Shaffer  (1986)  sequential  Bonferroni 
pairwise  comparisons  (familywise  a  =  .05)  indicated  that  both  label  groups  outperformed  the  No 
Label  group,  both  ps  <  .05.  No  reliable  performance  difference  was  found  between  the  two  label 
groups  (p=  .13). 

Discussion 

The  transfer  performance  of  the  three  training  groups  reveals  that  the  presence  of  a  label 
facilitates  performance  on  novel  problems.  These  results  are  consistent  with  the  hypothesis  that  a 
label  helps  learners  form  a  subgoal  for  the  labeled  steps.  Even  a  relatively  meaningless  label 
appears  to  promote  subgoal  learning,  presumably  because  it  serves  as  a  cue  to  learners  to  retrieve 
information  from  long-term  memory  in  order  to  explain  why  a  set  of  steps  belong  together. 

It  is  proposed  that  a  label  aids  transfer  because  it  leads  learners  to  group  a  set  of  steps  and 
then  to  self-explain  why  the  steps  go  together.  The  result  of  this  self-explanation  process  is  the 
subgoal.  The  results  from  Experiment  1,  while  demonstrating  the  relationship  between  labels  and 
improved  transfer,  did  not  directly  test  the  hypothesized  links  of  the  model.  Experiment  2  began  a 
more  direct  exploration  of  the  links  between  labeling,  grouping,  and  subgoal  formation. 
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Experiment  2 

Experiment  2  examined  the  hypothetical  connections  among  labeling,  grouping,  and 
subgoal  fonnation,  as  well  as  continuing  to  examine  the  relationship  between  labeling  and  transfer 

performance,  through  the  use  of  three  tasks.  The  first  task  was  a  segmenting  task  in  which 

participants  circled  the  steps  in  a  woiked-cxample  that  they  believed  went  together.  It  was 
hypothesu»d  that  participants  whose  examples  provided  a  label  for  the  set  of  steps  for  finding  the 
total  fiequency  would  be  more  likely  to  circle  those  steps  as  a  unit,  that  is,  to  group  those  steps, 
compared  to  the  No  Label  participants  who  might  be  more  likely  to  circle  the  entire  set  of  steps  for 

finding  X  as  a  unit. 

The  second  task  required  participants  to  provide  a  description  of  how  to  solve  problems  m 
-the  domain.  It  was  hypothesized  that  if  a  person  learns  a  subgoal,  such  as  the  subgoal  to  find  a 
total,  then  he  or  she  would  be  more  likely  to  mention  it  in  a  description  compared  to  a  learner  who 
did  not  leam  that  subgoal.  That  is,  the  description  would  be  more  hierarchically  organized 

compared  to  one  that  simply  listed  steps.2 

The  third  task  was  transfer  performance  on  novel  problems. 

Thus,  participants'  segmentation  performance,  descriptions,  and  transfer  performance  were 
used  as  converging  measures  of  subgoal  learmng. 

Method 

Participants.  Participants  were  90  students  recruited  from  an  introductory  psychology  class 
at  the  Georgia  Institute  of  Technology  who  received  course  credit  for  their  participation.  None  of 
the  students  had  taken  a  probability  course  prior  to  participating  in  the  experiment. 

Procedure.  Participants  studied  the  same  cover  sheet  as  in  Experiment  1  and 

were  randomly  assigned  to  one  of  three  groups.  The  Meaningful  Label  (N  -  30)  and  Less 
Meaningful  Label  (N  =  30)  groups  studied  the  same  three  examples  as  the  corresponding  groups  in 
Experiment  1  (in  which  the  steps  for  finding  the  total  frequency  were  expUcitly  labeled)  while  the 
No  Label  group  (M  =  30)  studied  the  same  examples  used  by  the  No  Label  group  m  Experiment  1. 
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The  solutions  studied  by  the  label  groups  were  modified  from  Experiment  I's  so  that  the 
steps  for  finding  the  total  frequency  were  on  the  same  line  as  the  rest  of  the  steps  for  finding  X, 
thus  making  the  solutions  more  visuaDy  similar  to  those  studied  by  the  No  Label  group  (see  Table 
3  for  an  example).  There  were  two  reasons  for  the  modification.  The  first  was  to  reduce  the 
chance  that  any  segmenting  and  transfer  differences  could  be  due  to  factors  other  than  labeling. 
The  second  was  to  guard  against  the  possibility  that  participants  in  the  label  groups  would  circle  a 
series  of  steps  simply  because  they  were  on  their  own  line. 


Insert  Table  3  about  here 


Accompanying  the  third  example  were  instructions  asking  participants  to  circle  steps  in  the 
solution  that  they  felt  formed  a  unit.  The  actual  instructions  were: 

In  the  solution  presented  below,  please  circle  the  groups  of  steps 
that  you  feel  go  together.  For  instance,  suppose  you  were  following 
a  recipe  for  cooking  something.  Perhaps  the  first  three  steps  of  the 
recipe  involved  putting  various  ingredients  into  a  bowl  and  the 
fourth  step  involved  stirring  the  ingredients  with  a  spoon  and  the 
fifth  step  involved  using  a  blender  to  finishing  the  mixing.  You 
might  draw  a  circle  around  the  first  three  steps  because  they  involve 
"adding  ingredients"  and  you  might  draw  a  circle  around  the  the 
fourth  and  fifth  steps  because  they  involve  "blending  the 
ingredients." 

After  performing  the  segmenting  task,  participants  were  asked  to  describe  how  to  solve 
problems  in  the  domain.  The  instructions  were; 
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Suppose  you  were  going  to  teach  someone  how  to  solve  Poisson 

distribution  problems  of  the  types  you  have  just  studied.  Please 

describe  the  procedure  or  procedures  you  would  give  someone  to 
solve  these  problems.  Please  be  as  complete  as  possible. 

After  writing  their  descriptions,  participants  solved  the  same  six  test  problems  used  in 
Experiment  1 .  Participants  were  told  not  to  look  back  at  the  examples  when  writing  their 
descriptions  or  solving  the  test  problems. 

Participants'  segmenting  performance  was  scored  for  whether  they  circled  the  steps  for 
finding  the  total  frequency  as  a  unit.  Participants'  explanations  of  how  to  solve  problems  m  the 
domain  were  scored  for  two  features:  an  expUcit  mention  of  trying  to  find  the  total  and  an  expUcit 
mention  of  trying  to  find  an  average.  Participants'  written  solutions  to  the  test  problems  were 
scored  for  whether  they  found  X  correctly. 

Two  raters  independently  scored  the  explanations  and  agreed  on  scormg  94%  of  the  time. 

Disagreements  were  resolved  by  discussion. 

Design.  The  independent  variable  was  type  of  example  solutions  studied  (Meaningful 
Label,  Less-Meaningful  Label,  No  Label),  thus  there  were  three  groups  in  the  experiment.  The 
dependent  measures  were  segmenting  performance,  descriptions  for  how  to  solve  the  problems, 
and  transfer  performance  on  the  six  test  problems. 

Predictions 

R«.fin<»nting.  If  the  label  manipulation  made  learners  more  likely  to  group  a  set  of  steps, 
then  the  label  groups  should  tend  to  circle  the  steps  (in  the  third  example)  for  finding  total 
frequency  as  a  single  unit  more  often  than  No  Label  participants. 

TVerriptinn  nf  Hnw  tn  Solve  Problems.  Since  it  was  hypothesized  that  the  label  groups 

would  be  more  likely  than  the  No  Label  group  to  leam  the  subgoal  of  finding  a  total,  the  label 
groups  should  be  more  likely  to  mention  the  idea  of  finding  a  total  in  their  descriptions  of  how  to 
solve  Poisson  problems.  The  Less-Meaningful  Label  group  might  mention  the  notion  of  finding 
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"fl"  rather  than  finding  the  "total  number"  of  things  if  there  is  some  tendency  by  learners  to  repeat 
the  wording  fixrm  examples.  All  groups  were  expected  to  mention  finding  X  equally  often  since 

this  subgoal  was  always  labeled  in  the  examples. 

Transfer.  As  in  Experiment  I ,  the  label  groups  were  predicted  to  be  more  likely  to  find  X 

successfully  on  novel  test  problems  since  they  involved  new  ways  of  finding  the  total  frequency. 
All  groups  were  predicted  to  find  X  correctly  in  the  isomorphic  test  problems  since  the  same  sets  of 

steps  used  in  the  examples  could  be  applied  to  those  problems. 

-Results 

Segmenting.  As  predicted,  the  label  groups  circled  the  steps  for  finding  total  frequency  as 
£a  single  unit  more  frequently  than  the  No  Label  group,  x^(2)  =  6.57,  p  =  .037  (see  Table  4). 

neQrriptinns  of  Hnw  to  Solve  Problems.  As  expected,  there  was  no  significant  difference 
among  the  groups  in  the  frequency  of  mentioning  the  subgoal  of  finding  X  or  the  average,  X^(2)  - 
0.69,  p  =  .71  (see  Table  4). 

The  fiequency  with  which  participants  mentioned  the  subgoal  of  finding  a  total  was 
analyzed  in  two  ways.  In  both  analyses,  participants  in  the  Less-Meaningful  Label  group  who 
mentioned  the  notion  of  finding  Q  and  also  explicitly  called  this  value  a  total  were  counted  in  Table 

4  as  having  mentioned  a  total  rather  than 

The  first  analysis  used  the  Total  vs.  Q  vs.  Neither  breakdown  in  Table  4  and  found  a 
significant  difference  among  the  groups,  X^(4)  =  28.14,  p  <  .0001.  The  percentages  in  Table  4 
suggest  that  the  label  groups  were  more  likely  to  mention  the  subgoal  of  finding  a  total  or  Q 
compared  to  the  No  Label  group.  In  the  second  analysis  the  Less-Meaningful  Label  participants 
who  mentioned  finding  Q  but  did  not  also  call  it  a  total  (N  =  8)  were  placed  in  the  same  category  as 
those  who  expliciUy  mentioned  a  total.  Thus,  all  participants  were  categorized  into  one  of  two 
groups:  those  mentioning  a  total  or  Q  versus  those  mentioning  neither.  Once  again  there  was  a 
significant  difference  between  the  groups  in  the  frequency  of  mentioning  total  /  O,  x^(2)  =  14.52, 

p  =  0007. 
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Insert  Table  4  about  here 


Transfer.  Test  problems  were  scored  as  in  Experiment  1. 

All  participants  except  one  in  the  Meaningful  Label  group  solved  both  isomorphic  problems 
correctly. 

As  jModicted,  there  were  significant  differences  among  the  three  groups  with  respect  to 
finding  X  in  the  four  novel  test  problems.  E(2. 87)  =  14.93,  p  =  .017,  MSg  =  3.50,  with  means  of 
3.07, 2.80,  and  1.73  for  the  Meaningful  Label,  Less-Meaningful  Label,  and  No  Label  groups, 
respectively.  Shaffer  (1986)  sequential  Bonfeironi  pairwise  comparisons  indicated  that  both  label 
groups  outperfonned  the  No  Label  group,  both  ps  <  .05.  No  reliable  performance  difference  was 

found  between  the  two  label  groups  (p  =  .57). 

Transfer  to  Novel  Problems  as  a  Function  of  SeglPCming  Pgrform^£S-  Participants  who 

circled  the  steps  for  finding  total  frequency  as  a  single  unit  on  the  third  training  example  (H  =  50) 
outperformed  those  who  did  not  circle  those  steps  as  a  unit  (N  =  40)  with  respect  to  performance 
on  the  novel  test  problems,  E(l,  88)  =  62.72,  p  <  .0001,  MSg  -  3.09,  Ms  -  3.28  and  1.60, 

respectively. 

Transfer  a.s  a  Function  of  Descriptions.  There  was  a  significant  difference  m  transfer 
performance  for  novel  problems  as  a  function  of  whether  participants  mentioned  finding  a  total  05 
=  35),  (H  =  8),  or  neither  (H  =  47)  in  their  descriptions  (Total  =  3.31,  Q  =  3.50,  Neither  = 
1.79),  E(2, 87)  =  27.49,  p  =  .0004,  MSp  =  3.21.  Pairwise  comparisons  indicated  that 
participants  mentioning  a  total  or  Q  did  not  differ  ftom  each  other  in  transfer  performance,  and  that 
both  groups  outperformed  those  who  did  not  mention  either,  both  p's  <  .05  (although  the  small 
number  of  participants  mentioning  only  £2  makes  interpretations  of  comparisons  with  them 

tentative). 


Page  45 


Participants  who  mcntioDCd  finding  the  average  (H  =  59)  did  not  perfomi  differently  ftom 
those  who  did  not  mention  Ending  it  (S  =  31),  Ed,  88)  =  0.56,  p  =  .46,  MSs  =  3.75,  Ms  =  2.64 

and  2.32,  respectively. 

Discussion 

The  purpose  of  Experiment  2  was  to  determine  whether  more  direct  support  could  be  found 
for  the  hypothesized  connections  among  labeling,  grouping,  and  subgoal  formation.  The 
segmenting  results  provide  initial  support  for  the  link  between  labeling  and  grouping,  and  the 
description  results  support  the  connection  between  labeling  and  subgoal  formation.  Finally,  the 
transfer  results  are  consistent  with  the  claim  that  subgoal  formation  aids  transfer.  In  sum, 
converging  evidence  was  found  for  a  connection  between  labeling  and  subgoal  formation. 

While  results  from  the  first  two  experiments  are  consistent  with  the  "label  as  groupmg  cue* 
view,  the  generality  of  the  subgoal  formed  as  a  function  of  the  label  is  unclear.  That  is,  a  subgoal 
that  is  formed  in  response  to  a  label  that  makes  mention  of  superficial  features  in  the  example  rmght 
become  tied  to  those  features.  For  instance,  the  subgoal  formed  by  Meaningful  Label  participants 
in  the  first  two  experiments  might  have  been  "find  the  total  number  Qfpbjects."  Conversely,  a 
subgoal  formed  in  response  to  a  more  abstract  label  might  be  less  likely  to  be  tied  to  superficial 
features.  For  instance,  the  subgoal  formed  by  Uss-Meaningful  Label  participants  might  have  been 
"find  the  total."  This  latter  subgoal  is  more  general  and  closer  to  being  formally  correct.^  Most 
participants  in  Experiment  2  who  mentioned  finding  a  total  in  their  descriptions  usually  mentioned 
the  total  in  terms  of  objects  or  things.  However,  it  is  possible  that  this  surface  feature  tie  was 

strongest  for  participants  in  the  Meaningful  Label  condition. 

One  implication  of  forming  a  subgoal  that  is  tied  to  superficial  features  is  that  the  learner  is 

confusing  superficial  and  structural  features  of  the  domain.  A  way  to  test  this  possibility  is  to 
constnict  test  problems  that  systematically  manipulate  the  relationship  between  superficial  and 
structural  features  and  observe  the  degree  to  which  the  features  guide  learners'  performance. 

For  instance,  Ross  (1987, 1989)  provided  students  with  various  types  of  probability 
examples  to  study  such  as  problems  dealing  with  permutations  and  combinations.  The  permutation 
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cxan5)les  involved  people  picking  objects  in  a  certain  order.  Because  the  problems  involved 
people  picking  objects,  the  number  of  objects  in  the  problem  provided  the  starting  value  for  the 
^finminator  in  the  pomutation  equation.  Some  of  the  test  problems  involved  people  being 
assigned  to  objects.  In  these  cases  the  number  of  people  in  the  problem  provided  the  starting  value 
for  the  denominator.  However,  students  typically  placed  the  number  of  objects  in  the 
denominator.  That  is,  students  in  Ross'  studies  appeared  to  confuse  the  superficial  features  of 
Kiimans  and  objects  with  the  domain-relevant  features  of  choosing  and  chosen. 

With  respect  to  the  experimental  materials  used  in  the  present  study ,  most  learners,  at  least 
at  the  coUege  level,  were  assumed  to  be  sufficiently  sophisticated  to  generalize  "total  number  of 

briefcases."  The  generalization  that  might  be  formed  though  was  unclear.  One  possibility  was  that 

the  generalization  would  be  "total  number  of  objects"  if  the  examples  involved  humans  usmg 
objects.  Learners  forming  this  generalization  would  be  predicted  to  be  more  successful  solving 
^  novel  problems  that  require  the  total  number  of  objects  to  be  calculated  in  new  ways  compared  to 
..  learners  not  forming  this  generalization.  However,  given  that  this  generalization  is  still  tied  to  a 
superficial  feature,  objects,  these  learners  might  faU  to  solve  correcUy  a  novel  problem  that  required 
,the  number  of  humans  rather  than  objects  for  the  total.  Learners  studying  examples  with  the  less- 
meaningful  label  who  form  the  subgoal  for  finding  a  total  might  be  less  likely  to  have  this  subgoal 
tied  to  a  superficial  feature.  As  a  result,  these  learners  would  be  less  likely  to  make  mistakes  on 
novel  problems  that  switch  the  roles  of  humans  and  objects  from  their  roles  in  the  training 

examples. 

This  possibility  was  explored  in  Experiment  3. 

Experiment  3 

In  Experiment  3  participants  were  divided  into  the  same  groups  as  the  first  two 
experiments.  However,  additional  test  problems  were  created  in  which  the  total  frequency  was 
calculated  using  people  rather  than  objects. 
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Performance  predictions  varied  as  a  function  of  training  condition  and  type  of  test  problem. 
The  first  four  test  problems  were  isomorphic  to  the  training  examples.  The  first  pair  involved 
falriiiating  the  total  number  of  objects  in  order  to  find  X  and  the  second  pair  involved  calculating 
the  number  of  people.  Participants  were  expected  to  solve  the  first  pair  with  Uttle  difficulty. 

Participants  were  also  expected  to  solve  the  second  pair  successfully  even  though  they 
involved  a  reversal  in  the  roles  of  humans  and  objects.  The  reason  involves  two  types  of 
superficial  sirmlarity  that  seem  most  likely  to  affect  performance.  The  first  is  the  format  of  the 
numbers  in  the  examples  and  problems.  For  instance,  consider  the  problem  in  Table  5a  in  which 
the  roles  for  humans  and  objects  are  reversed  from  the  examples  (such  as  the  one  in  Table  1).  In 
-both  cases  there  are  a  series  of  numbers  (one  through  four  for  the  example  in  Table  1  and  one 
through  five  in  the  problem  in  Table  5a)  in  which  each  number  has  another  value  associated  with  it. 
Each  number  is  multipUed  by  its  associated  value  and  the  products  are  added.  This  format 
sinularity  aUows  a  number-matching  approach  that  has  been  shown  to  be  a  powerful  factor  m 

transfer  performance  (Novick&Holyoak,  1991).  In  the  present  case,  this  number-matching 

approach  would  produce  the  correct  answer.  The  second  type  of  superficial  similarity  is  the  roles 
of  humans  and  objects.  They  are  reversed,  compared  to  the  examples,  in  the  second  pair  of 
isomorphs.  This  reversal  could,  in  some  circumstances,  lead  learners  to  place  incorrect  values  mto 

an  equation. 

It  was  expected  that  the  fust  type  of  similarity  would  drive  performance  because  the 
isomorphic  problems  do  not  supply  obvious  candidates  to  place  in  the  equation  other  than  those 
produced  via  number-matching.  Tlius,  regardless  of  whether  a  solver  noticed  or  understood  the 
rote-reversal,  he  or  she  would  sttU  be  likely  to  foUow  a  number-matching  approach  and  thus, 
produce  the  correct  answer.  For  instance,  for  the  problem  in  Table  5a,  the  learner  could  sull 
foUow  the  training  procedure  of  multiplying  ■  1  ”  by  the  number  nearest  it,  "2"  by  the  number 
nearest  it,  and  so  on.  This  approach  would  also  be  consistent  with  prior  observations  that  learners 
tend  to  prefer  to  use  teamed  sequences  of  steps  when  they  ate  allowable  in  new  problems  even  if 
they  are  not  optimal  (cf.  Luchins,  1942;  Singley  &  Anderson,  1989,  p  99). 
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The  above  j^iproach  would  not  work  for  the  transfer  problems  that  involve  a  change  in 
steps  such  as  those  in  Tables  2  and  5b  and  5c.  There  is  no  way  to  cany  out  the  old  set  of  steps  to 
solve  these  problems.  It  is  assumed  that  in  the  process  of  tiying  to  decide  how  to  solve  these 
problems,  the  learner  will  be  more  likely  to  notice  features  of  the  problem  including  which 
numbers  are  associated  with  humans  and  which  are  associated  with  objects.  As  a  result,  the  solver 
with  a  subgoal  linked  to  superficial  features  might  be  more  likely  to  want  to  find  or  calculate  the 
total  based  on  objects  even  for  the  reversed-corrcspondence  problems. 

Based  on  the  above  analysis,  predicted  performance  on  the  novel  problems  varied  as  a 
function  of  group  and  role-correspondence.  Four  of  the  novel  problems  provided  the  total 
frequency  direcUy  (such  as  the  problem  in  Table  2b).  Two  of  the  problems  involved  objects 
-  providing  the  total  event  frequency  and  two  involved  humans  providing  the  total  event  frequency 
(see  Table  5b  for  an  example  of  the  latter  type).  It  was  predicted  that  on  the  first  pair  of  problems 
both  label  groups  would  outperform  the  No  Label  group.  It  was  predicted  that  on  the  second  pair 
of  problems  Meaningful  Label  participants  would  be  more  likely  than  Less-Meaningful  Label 
participants  to  incorrectiy  place  the  number  of  objects  in  the  numerator  of  the  fraction  in  order  to 
..find  This  prediction  was  made  because  of  the  hypothesis  that  Meaningful  Label  participants 
would  be  more  likely  than  Less-Meaningful  Label  participants  to  associate  objects  with  finding  a 
total.  The  value  placed  in  the  numerator  of  the  fraction  presumably  represents  the  value  that  the 

participant  believes  is  the  total. 

The  last  four  problems  involved  adding  simple  frequencies  in  order  to  find  a  total  frequency 
(such  as  the  problem  in  Table  2a).  Two  of  the  problems  involved  objects  being  used  to  calculate 
the  total  event  frequency  and  two  involved  humans  being  used  to  calculate  the  total  event  frequency 
(see  Table  5c  for  an  example  of  the  latter  type).  It  was  predicted  that  on  the  first  pair  of  problems 
the  label  groups  would  outperform  the  No  Label  group.  The  second  pair  of  problems  provided 
two  sets  of  numbers.  The  first  set  of  numbers  could  be  added  to  produce  a  total  number  of  objects 
and  the  second  set  could  be  added  to  produce  a  total  number  of  people.  If  only  one  set  was 
provided,  then  participants  would  be  more  procedurally  constrained  (as  with  the  isomorphs)  and 
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there  would  be  less  of  a  chance  of  finding  a  performance  difference  between  Meaningful  and  Less- 
Meaningful  Label  participants.  It  was  predicted  that  on  the  second  pair  of  problems  the  Meaningful 
Label  participants  would  be  more  likely  than  Less-Meaningful  Label  participants  to  make  the 
mistakfi  of  calculating  a  total  using  objects  rather  than  humans. 

Thus,  for  "reversed  correspondence"  novel  test  problems,  it  was  predicted  that  Less- 
Meaningful  Label  participants  would  show  less  of  a  decrement  in  performance  than  Meaningful 
Label  participants,  relative  to  the  groups'  performance  on  the  "same-correspondence"  problems. 


Insert  Table  5  about  here 


Method 

Participants.  Participants  were  90  students  recruited  from  an  introductory  psychology  class 
at  the  Georgia  Institute  of  Technology  who  received  course  credit  for  their  participation.  None  of 
the  participants  had  taken  a  probability  course  prior  to  participating  in  the  experiment. 

Materials  and  Procedure.  The  conditions,  training  procedure,  and  cover  sheet  were 
identical  to  those  in  the  prior  experiments  and  there  were  30  participants  in  each  condition.  The 
examples  used  the  format  created  in  Experiment  2  that  made  the  visual  appearance  of  the  solutions 
used  in  the  label  conditions  more  similar  to  the  No  Label  solution. 

A  larger  number  of  test  problems  (12)  were  used  than  in  the  previous  experiments.  The 
first  four  test  problems  were  isomorphic  to  the  training  examples.  The  first  two  involved  objects  in 
the  total  frequency  and  the  next  two  involved  humans  in  the  total  frequency  (see  Table  5a  for  an 
example  of  the  latter  set).  The  next  four  test  problems  involved  the  total  frequency  being  given 
directly  in  the  problem.  The  first  pair  of  the  set  involved  objects  in  the  total  frequency  (see  Table 
2b  for  an  example)  while  the  second  pair  involved  humans  in  the  total  frequency  (see  Table  5b  for 
an  example).  The  next  four  test  problems  involved  calculating  the  total  frequency  by  adding  a  set 
of  simple  frequencies.  The  first  pair  of  the  set  involved  objects  in  the  total  frequency  (see  Table  2a 
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for  an  example)  whUe  the  second  pair  involved  humans  in  the  total  frequency  (see  Table  5c  for  an 
example). 

Participants'  written  solutions  were  scored  for  whether  they  found  X.  correctly. 

Design.  The  between-subjects  variable  was  type  of  example  solutions  studied  (Meaningful 
Label,  Less-Meaningful  Label,  No  Label)  and  the  within-subjects  variable  was  correspondence  of 
Jbs  roles  of  hnmans  and  objects  in  the  test  problems  to  their  roles  in  the  examples.  The  dependent 
measure  was  performance  on  the  12  test  problems. 

Results  and  Discussion 

As  in  the  prior  experiments,  participants  were  given  a  score  of  1  for  a  given  problem  if  they 
found  X  correcdy  and  a  score  of  0  otherwise.  The  scores  for  Problems  1  and  2,  the  two  problems 
.that  were  isomorphic  to  the  training  examples  and  had  the  same  role-correspondence  of  humans 
and  objects  as  the  examples,  were  summed,  thus  creating  a  score  from  0-2  for  performance  on 
those  problems.  Similarly,  a  score  from  0-2  was  calculated  for  the  isomorphs  that  had  a  reversed 
role-correspondence  of  humans  and  objects  (Problems  3  &  4).  Finally,  a  score  from  0-4  was 
calculated  for  the  novel  test  problems  with  the  same  role-correspondence  as  the  examples 
.(Problems  5, 6, 9,  &  10)  and  a  score  from  0-4  was  calculated  for  the  novel  test  problems  with  a 

reversed  role-correspondence  (Problems  7, 8, 1 1,  &  12). 

As  expected,  all  groups  did  quite  weU  at  finding  X  on  test  problems  that  were  isomorphs  to 

the  training  examples  regardless  of  whether  the  roles  of  humans  and  objects  were  reversed  from 
the  exanqjles.  In  fact,  all  participants  solved  each  of  these  problems  correctly. 

Table  6  presents  fire  groups'  performance  on  the  novel  test  problems  as  a  function  of 
whether  the  problems  involved  the  same  or  reversed  role-correspondence  of  humans  and  objects 
compared  to  the  training  examples.  An  analysis  of  variance  was  carried  out  on  the  performance  on 
the  novel  test  problems  with  group  as  the  between-subjects  variable  and  role-correspondence  (same 
as  examples  vs.  reversed  from  the  examples)  as  the  within-subject  variable. 

Consistent  with  the  results  from  the  prior  experiments,  there  was  a  significant  difference 
among  the  three  groups  with  respect  to  finding  X  on  the  novel  test  problems,  E(2, 87)  -  6.21,  p  - 


Page  51 


.003,  MSfi  =  5.70.  There  was  also  an  effect  of  role-correspondence  indicating  that  problems  with 
reversed  role-correspondence  were  solved  less  successfully  than  those  with  the  same  role- 
correspondence  as  the  examples.  E(l,  87)  =  26.56,  p  <  .0001/MSfe  =  0.86.  Finally,  there  was  a 
significant  interaction  between  group  and  role-correspondence,  E(2, 87)  =  6.25,  p  =  .003,  MSe  - 
0.86,  suggesting  that  the  correspondence  manipulation  affected  the  groups  differently. 

Separate  analyses  were  carried  out  for  each  group  comparing  performance  on  same  and 
reversed  role-correspondence  problems.  The  Shaffer  (1986)  procedure  for  providing  a  familywise 
.ffi  of  .05  for  multiple  comparisons  was  used.  The  Meaningful  Label  group  showed  a  significant 
decrease  in  performance  on  the  reversed  role-correspondence  problems  compared  to  the  same  role- 
, correspondence  problems.  E(l, 29)=  19.12.p=.0001.MS£=  1.54,  whUe  the  Less-Meaningful 
Label  and  No  Label  groups  did  not  show  significant  differences  in  performance  on  the  problem 
types  (ps  >  .05). 

Consistent  with  the  above  analysis,  if  performance  on  only  the  reversed  role- 
correspondence  problems  is  considered,  a  significant  effect  of  group  is  found,  E(2,87)  =  5.33.  p  = 
.007,  MSe  =  3.41,  with  pairwise  comparisons  indicating  that  the  Less-Meaningful  Label  group 
outperformed  the  other  groups  (both  ps  <  .04)  but  the  Meaningful  Label  group  did  not  outperform 
the  No  Label  group  (p  >  .26). 

It  was  predicted  that  a  typical  mistake  made  by  the  Meaningful  Label  participants  m  solving 
the  reversed  role-correspondence  problems  would  be  to  put  or  calculate  a  value  for  total  number  of 
objects  in  the  numerator.  One  way  of  examining  the  likelihood  of  making  this  mistake  is  to 
examine  performance  on  reversed  role-correspondence  problems  by  participants  who  solved  the 
same  role-correspondence  problems  correctly.  This  approach  would  therefore  consider  only 
participants  who  demonstrated  the  ability  to  transfer  to  problems  that  involved  a  change  m 
procedure  relative  to  the  training  examples.  Thus,  mistakes  on  the  reversed  role-correspondence 
problems  would  presumably  be  due  to  mapping  problems  rather  than  other  sorts  of  transfer 

difficulties. 
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Of  tbe  eight  Meaningful  Label  participants  who  found  X  correctly  in  all  four  same  role- 
correspondence  problems,  five  of  them  put  objects  in  the  numerator  for  finding  X  in  the  reversed 
role-correspondence  problems.  Conversely,  of  the  seven  Less-Meaningful  Label  participants  who 
found  X  correctly  in  all  four  same  role-correspondence  problems,  only  two  of  them  put  objects  in 
the  numerator  for  finding  for  finding  X  in  the  reversed  role-correspondence  problems.  While  these 
,  numbers  are  too  smaU  to  achieve  statistical  significance,  their  pattern  is  consistent  with  the 
interpretation  that  the  Meaningful  Label  participants  who  were  able  to  adapt  the  solution  procedure 
from  the  examples  were  more  likely  to  be  misled  by  superficial  features  compared  to  the  analogous 

Less-Meaningfiil  Label  participants. 


Insert  Table  6  about  here 


It  is  possible  that  the  results  from  this  experiment,  rather  than  reflecting  learners  forming  a 
subgoal  associated  with  a  superficial  features,  are  due  to  learners  including  superficial  features  m 
the  steps  for  finding  the  total.  The  present  results  do  not  allow  these  possibUities  to  be  clearly 
disentangled.  The  data  most  relevant  to  examining  this  issue,  the  descriptions  provided  by 
learners,  tended  to  contain  a  mention  of  objects  at  both  the  step  level  and  at  the  subgoal  level. 
Nevertheless,  the  transfer  results  suggest  that  participants  receiving  meaningful  labels  associated 
superficial  features  and  mathematical  roles  more  often  than  participants  receivmg  less-meaningful 

labels. 

General  Discussion 

Students  frequently  leam  a  solution  procedure  as  a  series  of  steps  with  UtUe  or  no  higher- 
level  organization  (Reed,  Dempster.  &  Ettinger..  1985).  As  a  result,  while  they  can  solve  new 
problems  that  involve  the  same  steps  as  a  previously-studied  example,  they  have  difficulty  with 
problems  that  require  a  change  in  the  steps,  even  though  the  conceptual  structure  from  the  example 

to  the  problem  is  preserved. 
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A  guiding  assumption  of  the  present  research  is  that  transfer  performance  wiU  be  enhanced 
if  a  solution  procedure  is  structured  by  subgoals  and  a  method  for  achieving  each  one  rather  than 
just  a  single  linear  set  of  steps  for  the  entire  procedure.  Presumably  there  is  a  continuum  of 
structuredness  depending  on  the  number  of  subgoals  into  which  a  procedure  is  broken. 

A  solution  strategy  containing  a  single  goal  and  a  set  of  steps  that  occur  in  a  predetermined 
order  to  achieve  that  goal  is  similar  to  a  rote  strategy.  Such  a  strategy  is  not  particularly  flexible  m 
terms  of  supporting  transfer  to  novel  problems.  Nevertheless.  Singley  and  Anderson  (1989)  point 
-out  that  there  are  advantages  to  a  rote  procedure:  it  is  more  efficient  in  terms  of  number  of  rule 
firings  (if  one  were  to  model  the  procedure  with  production  rules)  and  a  rote  procedure  is  often 
■easier  to  leam  and  perform  than  a  procedure  that  has  more  structure  to  it.  Thus,  it  seems  plausible 
that  learners  will  tend  to  form  a  rote  procedure  unless  induced  to  do  otherwise.  Factors  that  may 
lead  learners  to  form  more  structured  or  hierarchical  representations  of  procedures  include  their 
abiUty  to  articulate  goals  that  sets  of  steps  are  achieving  (Chi  et  al.,  1989),  the  learner's  reasomng 
style  (e.g,  Dufresne  et  al..  1992),  and  the  type  of  training  materials  used  (e.g.,  Catrambone,  1994; 

Eylon  &  Reif,  1984). 

Based  on  prior  work  involving  instructional  manipulations  (e.g.,  Eylon  &  Reif,  1984, 
Smith  &  Goodman,  1984),  problem  solving  (Anzai  &  Simon,  1979),  and  categorization  (e.g., 
Wattenmaker  et  al..  1986),  I  hypothesized  that  learners  would  be  more  likely  to  leam  a  subgoal 
from  an  example  if  the  steps  for  achieving  that  subgoal  were  labeled.  A  label  was  predicted  to 
make  a  learner  more  likely  to  group  the  set  of  steps  and,  through  a  self-explanation  process,  form  a 
subgoal  that  represented  the  purpose  of  the  steps.  The  focus  of  the  experiments  was  primarily  on 
the  connection  between  labeling,  subgoal  formation,  and  transfer  performance.  The  self¬ 
explanation  process  was  not  directly  examined. 

Experiments  1  and  2  demonstrated  that  the  pisssnES  of  a  label,  rather  than  its  semantic 
content,  can  be  sufficient  to  induce  a  learner  to  form  a  subgoal.  Participants  with  less-meaningful 
labels  were  able  to  solve  transfer  problems  as  successfully  as  participants  with  meaningful  labels 
and  both  groups  transferred  better  than  a  No  Label  group.  In  addition,  the  segmentation  task 
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results  were  consistent  with  the  claim  that  a  1^1  can  aid  grouping.  These  results,  coupled  with  the 
description  data  and  transfer  performance,  support  the  hypothesized  links  between  labeling, 
grouping,  and  subgoal  formation. 

Experiment  3  demonstrated  that  the  subgoal  formed  in  response  to  a  label  may  be  less  tied 
to  superficial  details  of  examples  when  the  label  does  not  reference  those  details.  Learners 
,  receiving  examples  using  less-meaningful  labels  transferred  more  successfully  than  other  learners 
to  novel  problems  that  altered,  with  respect  to  the  examples,  the  role-correspondence  between 
superficial  features  and  the  solution  procedure.  This  result,  besides  again  supporting  the  view  that 
labels  serve  as  a  grouping  cue,  suggests  that  the  generality  of  the  procedure  formed  from  examples 
can  be  increased  through  the  use  of  labels  that  do  not  contain  references  to  superficial  features  of 

examples. 

A  related  explanation  for  the  obtained  results  is  that  the  Less-Meaningful  Label  participants 
had  to  discover  the  purpose  of  the  labeled  steps  as  opposed  to  being  told.  The  strategies  or 
processes  involved  in  determining  the  purpose  of  the  steps  might  be  related  to  the  processes  for 
constructing  new  steps  or  modifying  old  steps  to  achieve  the  same  purpose  or  subgoal  (e.g., 
McDaniel  &  Schlager,  1990). 

Instructional  and  Individual  Differences  Factors  AffcftiPg  StfbSQal  Lgammg 

To  be  sure,  the  background  of  a  learner  plays  a  role  in  how  likely  he  or  she  is  to  leam  a 
subgoal.  Ausubel  (1968,  p  148-149)  suggested  that  the  value  of  "organizers"  hinges  upon  the 
learner  possessing  relevant  background  information  so  that  the  pieces  of  information  bemg 
organized  already  have  some  meaning.  For  instance,  if  a  student  learning  mechanics  is  told  that 
one  part  of  a  solution  procedure  is  to  determine  the  components  of  force  along  the  x  and  y  axes, 
this  organizer  for  the  subsequent  steps  will  be  of  minimal  use  if  the  learner  knows  little  or  nothing 
about  coordinate  systems  or  trigonometry. 

With  respect  to  the  present  study,  a  learner  with  a  weak  math  background  may  look  at  a 
series  of  addition  and  multiplication  steps  labeled  with  Q  and  not  group  them  or,  m  grouping  them, 
not  realize  that  the  steps  calculate  a  total.  This  learner  might  be  predicted  to  be  less  likely  to  form 
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the  subgoal  of  finding  the  total  number  of  objects  in  this  situation  compared  to  one  in  which  the 
steps  were  labeled  with  "total  number  of  briefcases  owned."  For  this  learner,  meaningful  labels 
might  produce  better  subgoal  learning  since  the  extra  domain  information  provided  by  a  meaningful 
label  could  help  the  learner  make  sense  of  the  steps  and  to  understand  their  purpose  (even  though 
the  resulting  subgoal  might  have  ties  to  superficial  features).  Conversely,  a  learner  with  a  stronger 
mafh  background  might  be  expected  to  recognize  that  the  series  of  steps,  when  separated  from  the 
other  steps  in  the  overall  solution  procedure  through  the  use  of  a  label  such  as  Q,  calculates  the 
total.  A  label  tied  to  superficial  features  of  the  problem  could  influence  this  learner  to  form  a  less- 
general  subgoal. 

Most  students  in  the  present  study  had  at  least  two  college-level  math  courses,  thus  it  is 
^  reasonable  that  when  cued  with  a  label  they  could  determine  that  a  set  of  steps  calculated  a  total. 

No  Label  students  were  less  likely  to  form  the  subgoal  of  finding  a  total,  presumably  because  it 
was  more  difficult  for  them  to  parse  the  solution  procedure  effectively.  Thus,  an  interesting  issue 
to  examine  in  future  work  would  be  the  effects  of  the  learners’  background  and  the  labels  used  in 
examples  on  subgoal  learning  and  transfer  performance. 

Other  Types  of  Labels 

While  learners  receiving  the  less-meaningful  label  were  more  likely  to  form  representations 
free  of  erroneous  superficial  features,  it  is  interesting  to  consider  what  the  nature  of  the 
representations  might  be  if  the  labels  were  meaningful,  but  unconnected  to  superficial  details  of  the 
examples.  For  instance,  instead  of  containing  the  label  "total  number  of  briefcases  owned," 
suppose  an  example  contained  the  label  "total  frequency  of  the  event."  This  label  is  formally 
correct  and  not  related  to  superficial  details  of  the  example.  Perhaps  this  sort  of  label  would 
produce  the  best  transfer  since  it  would  supply  a  formally  correct  concept,  presumably  cue  the 
grouping  and  self-explanation  processes,  and  not  provide  a  misleading  tie-in  to  superficial  features. 
On  the  other  hand,  it  is  possible  that  this  sort  of  label,  because  it  has  some  meaning  to  the  learner 
(at  least  relative  to  Q),  but  is  presumably  also  a  relatively  unfarmliar  concept,  could  distract  the 
learner  and  compete  with  the  grouping  and  self-explanation  processes.  This  is  an  interesting 
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empirical  question  that  could  be  examined  in  future  work  and  may  be  related  to  Sweller  and 
Cooper's  (1985)  conjecture  that  schemas  are  more  likely  to  be  learned  when  other  processes  do  not 
place  an  additional,  distracting  or  concurrent  load  on  working  memory  (see  also  Ward  &  Sweller, 
1990). 

Ts  There  a  "Right"  or  "Basic"  Level  of  Subgoal  Learmn£2 

A  solution  procedure  can  be  broken  down  into  any  number  of  subgoals  and  methods.  It 
is  unlikely  that,  with  respect  to  improving  transfer,  a  particular  breakdown  could  be  defended  in 
any  formal  way  either  by  an  appeal  to  a  particular  cognitive  architecture  or  to  a  task  analysis  unless 
one  restricts  the  set  of  problems  to  be  considered  in  the  domain  so  that  the  pool  of  potentially 
usefiil  subgoals  can  be  identified  by  the  researcher.  Factors  such  as  differences  in  learners 
background  and  hypothesized  working  memory  load  as  a  function  of  a  particular  breakdown 
would  also  need  to  be  considered,  although  both  could  potentially  be  addressed  by  a  cognitive 

architecture  that  represents  and  tracks  these  factors. 

Subgoals  at  varying  levels  of  the  solution  structure  hierarchy  could  affect  transfer  success 
to  different  degrees.  For  instance,  in  Experiment  2,  transfer  to  novel  problems  was  not  predicted 
by  whether  or  not  participants  mentioned  the  goal  of  finding  A,  in  their  descriptions;  rather,  it  was 
associated  with  mentioning  the  goal  of  finding  a  total.  This  may  be  because  the  goal  to  find  X  is 
too  high-level;  it  does  not  sufficiently  constrain  the  learner's  search  for  appropriate  steps  to  modify 
or  create.  It  is  intriguing  to  consider  whether  the  inconsistency  in  the  problem  solving  literature 
concerning  difficulties  in  procedural  transfer  may  potentially  be  explained  through  a  subgoal 

"level"  analysis. 

Future  Work 

Besides  exploring  some  of  the  issues  raised  above,  an  additional  extension  to  the  present 
work  would  be  to  examine  the  link  between  grouping  and  self-explanation  more  directly  than  was 
done  in  the  present  study.  Approaches  such  as  encouraging  learners  to  talk  out  loud  while 
studying  examples  would  provide  a  more  direct  test  of  the  link  between  grouping  and  self- 


Page  57 


explanations.  This  approach  would  also  allow  an  examination  of  whether  factors  other  than  labels 
fan  affect  grouping  and  whether  grouping  reliably  leads  to  subgoal  formation. 

Another  extension  focuses  on  whether  subgoals  can  be  effectively  communicated  through 
declarative  statements  or  whether  they  are  best  learned  through  examples.  Chi  et  al.  (1989) 
suggested  learners  acquire  knowledge  through  a  self-explanation  process  that  they  might  not 
be  able  to  acquire  as  effectively  otherwise.  Examples  might  provide  the  best  environment  for 
learners  to  engage  in  the  self-explanation  process  as  well  to  integrate  the  resulting  knowledge  with 
the  domain-relevant  knowledge  they  already  possess  (Chi,  de  Leeuw,  Chiu,  &  LaVancher,  1994). 
.Thus,  it  would  be  interesting,  both  pedagogically  as  well  as  for  the  development  of  models  of 
r  learning,  to  examine  how  effectively  subgoals  are  learned  through  examples  versus  declarative 

text. 
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Footnotes 


iThe  Poisson  distribution  is  often  used  to  j^ioximate  binomial  probabilities  for  events 

occurring  with  some  small  probability.  The  Poisson  equation  is  .  where  X  is 

the  av^ge  (the  expected  value)  of  the  random  variable  X. 

2lt  is  possible  that  tte  segmentation  task  could  affect  performance  on  the  description  task 
'  by  l«v*^ing  participants  to  pay  more  attention  to  the  steps  than  they  might  otherwise.  However,  a 
prior  study  (Catrambone.  in  press)  found  a  relationship  between  a  similar  labeling  manipulation 
and  description  performance  without  the  intervening  segmentation  task. 

^The  most  formal  view  would  be  "total  fiequency  of  the  event." 
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Table  1 

Training  Example  with  Meaningful.  Less-Meaningful,  and  No  Label  SplytiQPS 

A  judge  noticed  that  some  of  the  219  lawyers  at  City  Hall  owned  more  than  one  briefcase.  She 
counted  the  number  of  briefcases  each  lawyer  owned  and  found  that  180  of  the  lawyers  owned 
exactly  1  briefcase,  17  owned  2  briefcases,  13  owned  3  briefcases,  and  9  owned  4  briefcases. 
Use  the  Poisson  distribution  to  determine  the  probability  of  a  randomly  chosen  lawyer  at  City 
Hall  owning  exactly  two  briefcases. 

a.)  No  Label  Solution: 

1(180) +  2(17)  + 3(13) -t- 4(9)  _  289 
219  219 

=  1.32  =  X  =  average  number  of  briefcases  owned  per  lawyer 

PPc=x,=l!iM 

x! 

rrx  2)  K2.718-*-%1.32^)]_  (.27X1.74)  _  235 
^  2!  2 


h.l  Meaningful  Label  Solution: 

Total  number  of  briefcases  owned  =  [1(180)  +  2(17)  +  3(13)  +  4(9)]  =  289 

E(X)  =  —  =  1.32  =  X  =  average  number  of  briefcases  owned  per  lawyer 
^  219 


P(X=x)  = 


x! 


P(X=2)  =  f(2  718'^-^^)(l-32^)]  ^  (■27X1.74)  ^  235 
^  2!  2 


c  )  T^ss-Meaningful  Label  Solution: 

£2  =  [1(180)  +  2(17)  +  3(13)  +  4(9)]  =  289 

E(X)  =  =  1.32  =  X  =  average  number  of  briefcases  owned  per  lawyer 

219 

P(X=x,JilM 

x! 

r(X-2)  -  l(2-718'^-^^)(l-32^)]  _  (■27)(1.74)  _  235 
2!  2 
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Table  2 

<SaTnplp,  Test  Problems 


a.)  Total  Frequency  Calculated  by  Adding  Simple  Frequencies 

Over  the  course  of  the  summer,  a  group  of  5  kids  used  to  walk  along  the  beach  each  day  collecting 
seashells.  We  know  that  on  Day  1  Joe  found  4  shells,  on  Day  2  Sue  found  2  shells,  on  Day  3 
Mary  found  5  shells,  on  Day  4  Roger  found  3  shells,  and  on  Day  5  Bill  found  6  shells.  Use  the 
Poisson  distribution  to  determine  the  probability  of  a  randomly  chosen  kid  finding  3  sheUs  on  a 

particular  day. 

Solution  (not  seen  by  participants); 

_  4  +  2  +  5  +  3-<-6  _  ^  _  4  0  =  X  =  average  number  of  shells  per  kid 
5  5 

..  [(2.718-»  »)(4.0^]  _  (.OISKM)  .  )o. 


b.)  Total  Frequency  Provided  Directly 

A  number  of  celebrities  were  asked  how  many  commercials  they  made  over  the  last  year.  The  20 
celebrities  made  a  total  of  71  commercials.  Use  the  Poisson  distribution  to  determine  the 
probability  that  a  randomly  chosen  celebrity  made  exactly  5  commercials. 


Solution  (not  seen  by  participants); 


71 

E(X)  =  ^ 


=  3.55  =  X  =  average  number  of  commercials  per  celebrity 


_  ,,  [(2.718-3  ”)(3.55^)1  _  (.029)(563.8)  _ 

P(X=5)  =  120 
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(Hyperiment  2) 


Meaningful  Label  Group: 

_  1(180)  +  2(17)  +  3(13)  +  4(9)  _  total  number  of  briefcases  owned  _  2^ 

E(X)  =  - -  219  ~  219  219 

=  1 .32  =  Jl  =  average  number  of  briefcases  owned  per  lawyer 


P(X=x)  = 


x! 


P(X=2)  = 


1(2.718-‘-32)(i,322)]  (.27)(1.74)  _ 

2!  2 


.235 


Less-Meaningful  Label  Group: 


^  ia801 +  2(17) +  3(13)  + 4(9)  _ 

E(X)  =  ^ - “219  ^  219  219 

=  1.32  =  X,  =  average  number  of  briefcases  owned  per  lawyer 


- 


P(X=x)  = 


x! 


r(2.718->-32)(1.32^)]_(jZKl:Z£)=  235 

P(X=2)  s= - ”27  2 
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Table  4 

segmenting  and  Descriptions  as  a  Function  of  GrPMP  (Expgnmgpt  2.) 


Group 

Meaningful  Less-Meaningful  No 
Label  Label  Label 

(N  =  30)  (N  =  30)  (N  =  30) 


Percentage  Circling 
Total  Frequency 


63  67  37 


Percentage  Mentioning  Finding 


67 

60 

70 

Total 

57 

40 

20 

a 

0 

27 

0 

Neither 

43 

33 

80 
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Sample  Test  Problems  rExperimenLH 

a.)  Weighted-Average  with  Humans  Providing  Total  Frequency 

The  17  jq)rons  at  a  large  restaurant  were  each  worn  one  or  more  times  by  the  various  chefs.  Three 
of  the  aprons  were  each  used  by  exactly  1  chef,  7  of  the  aprons  were  used  by  2  of  the  chefs,  4  of 
the  aprons  were  used  by  3  of  the  chefs,  2  of  the  aprons  were  used  by  4  of  the  chefs,  and  1  of  the 
aprons  was  used  by  5.of  the  chefs.  Use  the  Poisson  distribution  to  determine  the  probability  of  a 
randomly  chosen  apron  being  worn  by  exactly  2  chefs. 

Solution  (not  seen  by  participants): 

-  K3)  +  2(7)  +  3(4)  +  4(2)  +  5(1)  _  42  _  ^  number  of  chefs  per  apron 

r(2.718-2-^7)(2.472)]  _  (.085)(6.1)  _ 
nA-Z)  -  2!  2  ■ 


b.)  Total-Frequency-Given-Directly  with  Humans  Providing  Total  Frequency 

Over  a  period  of  time  at  a  certain  video  store,  243  people  rented  104  different  videos.  Use  the 
Poisson  distribution  to  determine  the  probability  that  a  randomly  chosen  video  was  rented  exactly  4 

times. 


Solution  (not  seen  by  participants): 


E(X)  = 


—  =  2  34  =  X  =  average  number  of  renters  per  video 
104 


P(X=4)  = 


1(2.7 1  8-^-3^)(2.34^)1  (.096)(29.98) 

4!  ”  24 


.12 


(Table  5  continued  on  the  next  page) 


Table  5  (con't) 


c.)  Single-Frequency  with  Humans  Providing  Total  Frequency 

An  accounting  firm  employing  many  accountants  woiked  on  a  large  number  of  tax  returns  and 
used  many  types  of  tax  forms.  Four  of  the  accountants  were  interviewed  and  it  was  found  that  one 
woiked  on  3  tax  returns  that  day,  another  woiked  on  9,  a  third  worked  on  5,  and  the  fourth 
woiked  on  6.  In  addition,  of  the  many  different  types  of  tax  forms  used,  it  was  found  that  one 
type  of  tax  form  was  used  by  12  accountants  at  the  firm,  another  type  was  used  by  8  accountants,  a 
third  type  was  used  by  6  accountants,  and  a  fourth  type  was  used  by  9  accountants.  Use  the 
Poisson  distribution  to  determine  the  probability  of  a  randomly  chosen  type  of  tax  form  being 
worked  on  by  7  different  accountants. 


Solution  (not  seen  by  participants); 


E(X)  =  ^  ^  =  8.75  =  X,  =  average  number  of  accountants  p)er  form 

^  '  4  4 


P(X=7) 


[(2.718-^'^^)(8.75^)]  (.00016)(3926960)  _  ^^5 


7! 


5040 
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Table  6 


Meaningful 

Label 

(N  =  30) 

Less-Meaningfiil 

Label 

(N  =  30) 

No 

Label 
(N  =  30) 

AVG 

Same  Correspondence 

3.07 

3.00 

1.53 

2.53 

Reversed  Correspondence 

1.67 

2.67 

1.13 

1.82 

AVG 

2.37 

2.83 

1.33 

2.18 

Note.  Maximum  possible  score  for  any  cell  -  4. 
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